Re: Changes to how EXPORTS are handled
Mike Hommey wrote: On Wed, Sep 04, 2013 at 11:35:19AM -0700, Gregory Szorc wrote: It's worth explicitly mentioning that tiers limit the ability of the build system to build concurrently. So, we have to choose between speed and a moving/complex target of dependency correctness. We have chosen to sacrifice this dependency correctness to achieve faster build speeds. If we want to keep ensuring dependency correctness, I believe we should accomplish that via static analysis, compiler plugins, standalone builds, or special build modes. Or not using -I$(DIST)/include. Some headers install into $(DIST)/include/mozilla and there's no other way to include them. -- Warning: May contain traces of nuts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Changes to how EXPORTS are handled
On Wed, Sep 04, 2013 at 12:31:02AM -0700, Gregory Szorc wrote: Assuming it sticks, bug 896797 just landed in inbound and changes how EXPORTS/headers are installed. This may impact your developer workflow if you modify EXPORTS in a moz.build file to add/remove headers. Previously, headers were installed incrementally as part of make directory traversal. In the new world, we write out a manifest of headers when the build config is read from moz.build files and then we install them in bulk at the top of the build. You'll see a new message like: From ./dist/include: Kept 3022 existing; Added/updated 0; Removed 678 files and 5 directories. This is the rsync-like process applying that manifest and reporting that 3022 files are unchanged and 678 files were unaccounted for, so they were purged from dist/include. The important developer change here is that a non top-level make will no longer install headers. So if you add a new .h to an EXPORTS in a moz.build file, you'll need to perform a top-level/full build for it to be installed. If you don't want to do a full top-level build, you can also just do make -C objdir export (or mach build export). Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Changes to how EXPORTS are handled
On Wednesday 2013-09-04 00:31 -0700, Gregory Szorc wrote: Assuming it sticks, bug 896797 just landed in inbound and changes how EXPORTS/headers are installed. This may impact your developer workflow if you modify EXPORTS in a moz.build file to add/remove headers. Previously, headers were installed incrementally as part of make directory traversal. In the new world, we write out a manifest of headers when the build config is read from moz.build files and then we install them in bulk at the top of the build. Does this undo the protection that the build tiers were designed for, which is to prevent backwards dependencies between parts of the build? (In other words, things like preventing XPCOM from depending on headers in layout, so that XPCOM could be used standalone.) Do we still care about ensuring this? If so, should we have some other mechanism (like having standalone builds and showing them on tbpl)? -David -- 턞 L. David Baron http://dbaron.org/ 턂 턢 Mozilla https://www.mozilla.org/ 턂 Before I built a wall I'd ask to know What I was walling in or walling out, And to whom I was like to give offense. - Robert Frost, Mending Wall (1914) signature.asc Description: Digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Changes to how EXPORTS are handled
On Wed, Sep 04, 2013 at 10:28:21AM +0100, L. David Baron wrote: On Wednesday 2013-09-04 00:31 -0700, Gregory Szorc wrote: Assuming it sticks, bug 896797 just landed in inbound and changes how EXPORTS/headers are installed. This may impact your developer workflow if you modify EXPORTS in a moz.build file to add/remove headers. Previously, headers were installed incrementally as part of make directory traversal. In the new world, we write out a manifest of headers when the build config is read from moz.build files and then we install them in bulk at the top of the build. Does this undo the protection that the build tiers were designed for, which is to prevent backwards dependencies between parts of the build? (In other words, things like preventing XPCOM from depending on headers in layout, so that XPCOM could be used standalone.) Do we still care about ensuring this? If so, should we have some other mechanism (like having standalone builds and showing them on tbpl)? The way the tier build works is that we effectively make export in all directories of a same tier before make libs. In practice, this means xpcom had access to every header in platform already, and any tier built before platform, for that matter. So only app headers weren't available to xpcom, and that's not a lot of them. So, really, nothing was already there to prevent backwards dependencies. At least not in a very long time (I don't remember if we ever did (make export; make libs) recursively directory by directory instead of tier by tier.) Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Changes to how EXPORTS are handled
On Wednesday 2013-09-04 18:45 +0900, Mike Hommey wrote: On Wed, Sep 04, 2013 at 10:28:21AM +0100, L. David Baron wrote: On Wednesday 2013-09-04 00:31 -0700, Gregory Szorc wrote: Assuming it sticks, bug 896797 just landed in inbound and changes how EXPORTS/headers are installed. This may impact your developer workflow if you modify EXPORTS in a moz.build file to add/remove headers. Previously, headers were installed incrementally as part of make directory traversal. In the new world, we write out a manifest of headers when the build config is read from moz.build files and then we install them in bulk at the top of the build. Does this undo the protection that the build tiers were designed for, which is to prevent backwards dependencies between parts of the build? (In other words, things like preventing XPCOM from depending on headers in layout, so that XPCOM could be used standalone.) Do we still care about ensuring this? If so, should we have some other mechanism (like having standalone builds and showing them on tbpl)? The way the tier build works is that we effectively make export in all directories of a same tier before make libs. In practice, this means xpcom had access to every header in platform already, and any tier built before platform, for that matter. So only app headers weren't available to xpcom, and that's not a lot of them. So, really, nothing was already there to prevent backwards dependencies. At least not in a very long time (I don't remember if we ever did (make export; make libs) recursively directory by directory instead of tier by tier.) So I was assuming that xpcom was in a different tier from layout; apparently that's not the case. (I thought it was originally, but my memory could be wrong.) But is it correct that this change means we no longer have the backwards-dependency checking for things in different tiers? -David -- 턞 L. David Baron http://dbaron.org/ 턂 턢 Mozilla https://www.mozilla.org/ 턂 Before I built a wall I'd ask to know What I was walling in or walling out, And to whom I was like to give offense. - Robert Frost, Mending Wall (1914) signature.asc Description: Digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Changes to how EXPORTS are handled
On 9/4/13 3:04 AM, Mike Hommey wrote: On Wed, Sep 04, 2013 at 10:57:26AM +0100, L. David Baron wrote: The way the tier build works is that we effectively make export in all directories of a same tier before make libs. In practice, this means xpcom had access to every header in platform already, and any tier built before platform, for that matter. So only app headers weren't available to xpcom, and that's not a lot of them. So, really, nothing was already there to prevent backwards dependencies. At least not in a very long time (I don't remember if we ever did (make export; make libs) recursively directory by directory instead of tier by tier.) So I was assuming that xpcom was in a different tier from layout; apparently that's not the case. (I thought it was originally, but my memory could be wrong.) You're right, this was changed in bug 528250 http://hg.mozilla.org/mozilla-central/rev/b7b5c61b1af8 But is it correct that this change means we no longer have the backwards-dependency checking for things in different tiers? Yes, but since most things are in the platform tier, and the platform tier is almost last (there's only the app tier after it), it wasn't much relevant. What probably matters most to protect is js, and js standalone builds should provide that protection. It's worth explicitly mentioning that tiers limit the ability of the build system to build concurrently. So, we have to choose between speed and a moving/complex target of dependency correctness. We have chosen to sacrifice this dependency correctness to achieve faster build speeds. If we want to keep ensuring dependency correctness, I believe we should accomplish that via static analysis, compiler plugins, standalone builds, or special build modes. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Changes to how EXPORTS are handled
On Wed, Sep 04, 2013 at 11:35:19AM -0700, Gregory Szorc wrote: On 9/4/13 3:04 AM, Mike Hommey wrote: On Wed, Sep 04, 2013 at 10:57:26AM +0100, L. David Baron wrote: The way the tier build works is that we effectively make export in all directories of a same tier before make libs. In practice, this means xpcom had access to every header in platform already, and any tier built before platform, for that matter. So only app headers weren't available to xpcom, and that's not a lot of them. So, really, nothing was already there to prevent backwards dependencies. At least not in a very long time (I don't remember if we ever did (make export; make libs) recursively directory by directory instead of tier by tier.) So I was assuming that xpcom was in a different tier from layout; apparently that's not the case. (I thought it was originally, but my memory could be wrong.) You're right, this was changed in bug 528250 http://hg.mozilla.org/mozilla-central/rev/b7b5c61b1af8 But is it correct that this change means we no longer have the backwards-dependency checking for things in different tiers? Yes, but since most things are in the platform tier, and the platform tier is almost last (there's only the app tier after it), it wasn't much relevant. What probably matters most to protect is js, and js standalone builds should provide that protection. It's worth explicitly mentioning that tiers limit the ability of the build system to build concurrently. So, we have to choose between speed and a moving/complex target of dependency correctness. We have chosen to sacrifice this dependency correctness to achieve faster build speeds. If we want to keep ensuring dependency correctness, I believe we should accomplish that via static analysis, compiler plugins, standalone builds, or special build modes. Or not using -I$(DIST)/include. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
