Re: Thinking about the merge with unified build
On 18:23, Mon, 02 Dec, Ehsan Akhgari wrote: As for identifying broken non-unified builds, can we configure one of our mozilla-inbound platforms to be non-unified (like 32-bit Linux Debug)? I think the answer to that question depends on how soon bug 942167 can be fixed. Chris, any ideas? We're trying to figure out the best way to implement it. It'll be a week or so at least. signature.asc Description: Digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Mitigating unified build side effects Was: Thinking about the merge with unified build
Mike Hommey wrote:
On Sat, Nov 30, 2013 at 12:39:59PM +0900, Mike Hommey wrote:
Incidentally, in those two weeks, I did two attempts at building
without unified sources, resulting in me filing 4 bugs in different
modules for problems caused by 6 different landings[1]. I think it is time
to seriously think about having regular non-unified builds (bug 942167).
If that helps, I can do that on birch until that bug is fixed.
Speaking of which, there are essentially two classes of such errors:
- missing headers.
- namespace spilling.
The latter is due to one source doing using namespace foo, and some
other source forgetting the same because, in the unified case, they
benefit from the other source doing it. I think in the light of unified
sources, we should forbid non-scoped use of using.
That is:
using namespace foo;
would be forbidden, but
namespace bar {
using namespace foo;
}
wouldn't. In most cases, bar could be mozilla anyways.
Thoughts?
Engineering-heavy solution:
* You can extend one of clang header checks(eg only include what you
use) to make sure each cpp actually includes what they use.
* ditto re writing a clang analysis for namespace collisions.
Some clever contributor like JCranmer could knock this out in no time.
We could then run these on debug builds on our infra without much extra
overhead.
Throw resources at this solution:
Alternatively, forcing a non-unified debug build on our infra would
catch most of these at higher infra cost.
Taras
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Thinking about the merge with unified build
On 11/29/13, 7:39 PM, Mike Hommey wrote: I think it's time, 9 days before the merge, to think about whether we want unified builds to ride the train or not. I'm almost tempted to suggest that we disable unified builds on nightlies, but that would probably hide the problems even more. I don't think Mike's original question has been discussed in this thread: should we disable unified builds for Aurora 28? I think it is a good idea because there are still open questions about unified build code and stability issues (though Aurora will then hit the non-unified build problems right away). As for identifying broken non-unified builds, can we configure one of our mozilla-inbound platforms to be non-unified (like 32-bit Linux Debug)? cpeterson ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Thinking about the merge with unified build
On 12/2/2013, 2:36 PM, Chris Peterson wrote: On 11/29/13, 7:39 PM, Mike Hommey wrote: I think it's time, 9 days before the merge, to think about whether we want unified builds to ride the train or not. I'm almost tempted to suggest that we disable unified builds on nightlies, but that would probably hide the problems even more. I don't think Mike's original question has been discussed in this thread: should we disable unified builds for Aurora 28? I think it is a good idea because there are still open questions about unified build code and stability issues (though Aurora will then hit the non-unified build problems right away). I discussed this with Mike on IRC last Friday, and have thought about this a bit more since then, and I believe that we should disable unified builds on mozilla-aurora and all branches past that point, perhaps indefinitely. Unified builds don't really buy us that much for code that gets compiled on those branches anyway, and if developers want to have them locally then they can always pref it on in their mozconfigs. I filed this as bug 945496. As for identifying broken non-unified builds, can we configure one of our mozilla-inbound platforms to be non-unified (like 32-bit Linux Debug)? I think the answer to that question depends on how soon bug 942167 can be fixed. Chris, any ideas? Thanks! Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Mitigating unified build side effects Was: Thinking about the merge with unified build
On 12/2/2013, 2:26 PM, Taras Glek wrote:
Mike Hommey wrote:
On Sat, Nov 30, 2013 at 12:39:59PM +0900, Mike Hommey wrote:
Incidentally, in those two weeks, I did two attempts at building
without unified sources, resulting in me filing 4 bugs in different
modules for problems caused by 6 different landings[1]. I think it is
time
to seriously think about having regular non-unified builds (bug 942167).
If that helps, I can do that on birch until that bug is fixed.
Speaking of which, there are essentially two classes of such errors:
- missing headers.
- namespace spilling.
The latter is due to one source doing using namespace foo, and some
other source forgetting the same because, in the unified case, they
benefit from the other source doing it. I think in the light of unified
sources, we should forbid non-scoped use of using.
That is:
using namespace foo;
would be forbidden, but
namespace bar {
using namespace foo;
}
wouldn't. In most cases, bar could be mozilla anyways.
Thoughts?
Engineering-heavy solution:
* You can extend one of clang header checks(eg only include what you
use) to make sure each cpp actually includes what they use.
include-what-you-use currently has some edge cases that it cannot
properly track (such as the case where only macros from a header get
used, which breaks a common #include patterns such as
http://mxr.mozilla.org/mozilla-central/source/content/base/src/nsGkAtoms.h#30.
It has some other problems which I have hit before but I don't
remember them now. It's not entirely clear to me how easy these
problems will be to fix in IWYU.
* ditto re writing a clang analysis for namespace collisions.
Well technically in order to do something like that you'd just do the
semantic analysis phase and look at the compiler diagnostics. :-)
Some clever contributor like JCranmer could knock this out in no time.
We could then run these on debug builds on our infra without much extra
overhead.
Currently our static analysis infra only works on Linux. But note that
these two problems are not the only ones that we need to be worrying
about, and it's not clear to me whether it's worth for us to spend a lot
of effort in this analysis when a simple alternative exists (just
compiling the code in non-unified mode.)
Throw resources at this solution:
Alternatively, forcing a non-unified debug build on our infra would
catch most of these at higher infra cost.
See bug 942167.
Cheers,
Ehsan
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Mitigating unified build side effects Was: Thinking about the merge with unified build
Ehsan Akhgari wrote:
On 12/2/2013, 2:26 PM, Taras Glek wrote:
Mike Hommey wrote:
On Sat, Nov 30, 2013 at 12:39:59PM +0900, Mike Hommey wrote:
Incidentally, in those two weeks, I did two attempts at building
without unified sources, resulting in me filing 4 bugs in different
modules for problems caused by 6 different landings[1]. I think it is
time
to seriously think about having regular non-unified builds (bug
942167).
If that helps, I can do that on birch until that bug is fixed.
Speaking of which, there are essentially two classes of such errors:
- missing headers.
- namespace spilling.
The latter is due to one source doing using namespace foo, and some
other source forgetting the same because, in the unified case, they
benefit from the other source doing it. I think in the light of unified
sources, we should forbid non-scoped use of using.
That is:
using namespace foo;
would be forbidden, but
namespace bar {
using namespace foo;
}
wouldn't. In most cases, bar could be mozilla anyways.
Thoughts?
Engineering-heavy solution:
* You can extend one of clang header checks(eg only include what you
use) to make sure each cpp actually includes what they use.
include-what-you-use currently has some edge cases that it cannot
properly track (such as the case where only macros from a header get
used, which breaks a common #include patterns such as
http://mxr.mozilla.org/mozilla-central/source/content/base/src/nsGkAtoms.h#30.
It has some other problems which I have hit before but I don't remember
them now. It's not entirely clear to me how easy these problems will be
to fix in IWYU.
* ditto re writing a clang analysis for namespace collisions.
Well technically in order to do something like that you'd just do the
semantic analysis phase and look at the compiler diagnostics. :-)
Some clever contributor like JCranmer could knock this out in no time.
We could then run these on debug builds on our infra without much extra
overhead.
Currently our static analysis infra only works on Linux. But note that
these two problems are not the only ones that we need to be worrying
about, and it's not clear to me whether it's worth for us to spend a lot
of effort in this analysis when a simple alternative exists (just
compiling the code in non-unified mode.)
mac support would be straight-forward.
Throw resources at this solution:
Alternatively, forcing a non-unified debug build on our infra would
catch most of these at higher infra cost.
See bug 942167.
To clarify my earlier email: I think the right approach here is what you
are suggesting in above bug and if it turns out someone contributes
these fun-to-write analyses, we can switch.
Taras
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Disabling unified builds on mozilla-inbound debug builds Was: Thinking about the merge with unified build
On Mon, Dec 02, 2013 at 06:23:00PM -0500, Ehsan Akhgari wrote: On 12/2/2013, 2:36 PM, Chris Peterson wrote: On 11/29/13, 7:39 PM, Mike Hommey wrote: I think it's time, 9 days before the merge, to think about whether we want unified builds to ride the train or not. I'm almost tempted to suggest that we disable unified builds on nightlies, but that would probably hide the problems even more. I don't think Mike's original question has been discussed in this thread: should we disable unified builds for Aurora 28? I think it is a good idea because there are still open questions about unified build code and stability issues (though Aurora will then hit the non-unified build problems right away). I discussed this with Mike on IRC last Friday, and have thought about this a bit more since then, and I believe that we should disable unified builds on mozilla-aurora and all branches past that point, perhaps indefinitely. Unified builds don't really buy us that much for code that gets compiled on those branches anyway, and if developers want to have them locally then they can always pref it on in their mozconfigs. I filed this as bug 945496. As for identifying broken non-unified builds, can we configure one of our mozilla-inbound platforms to be non-unified (like 32-bit Linux Debug)? I think the answer to that question depends on how soon bug 942167 can be fixed. Chris, any ideas? In the meanwhile, I proposed in bug 945496 that we disable unified builds on all debug builds, until we have something better, because, while I'm running non-unified builds on the birch branch, at 7 days from the merge, the chances of non-unified builds to break are pretty big, especially when you consider three different bugs landed in the past 12 hours that broke them each in their own way. I will land the patch doing this later today or further tomorrow, after I'm sure the tree I'm landing on still builds this way (and I'd rather have that happen on top of a multiple-way fx-team and b2g-inbound merge) Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Mitigating unified build side effects Was: Thinking about the merge with unified build
On Sat, Nov 30, 2013 at 6:56 AM, Benoit Jacob jacob.benoi...@gmail.comwrote:
I'm all for reducing usage of 'using' and in .cpp files I've been switching
to doing
namespace foo {
// my code
}
instead of
using namespace foo;
// my code
This only works if you want to put all of the symbols in your file in the
'foo' namespace. In XPConnect, each file has stuff in the global namespace,
some stuff in namespace xpc, and a smattering of other namespaces. Putting
all of this in namespace JS doesn't make sense for obvious reasons, but we
still want to use all the JS symbols without prefixing.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Mitigating unified build side effects Was: Thinking about the merge with unified build
I'm all for reducing usage of 'using' and in .cpp files I've been switching
to doing
namespace foo {
// my code
}
instead of
using namespace foo;
// my code
where possible, as the latter leaks to other .cpp files in unified builds
and the former doesn't.
Regarding the proposal to ban 'using' only at root scope only, keep in mind
that we have conflicting *nested* namespaces too:
mozilla::ipc
mozilla::dom::ipc
so at least that class of problems won't be solved by this proposal. But I
still agree that it's a step in the right direction.
Benoit
2013/11/29 Mike Hommey m...@glandium.org
On Sat, Nov 30, 2013 at 12:39:59PM +0900, Mike Hommey wrote:
Incidentally, in those two weeks, I did two attempts at building
without unified sources, resulting in me filing 4 bugs in different
modules for problems caused by 6 different landings[1]. I think it is
time
to seriously think about having regular non-unified builds (bug 942167).
If that helps, I can do that on birch until that bug is fixed.
Speaking of which, there are essentially two classes of such errors:
- missing headers.
- namespace spilling.
The latter is due to one source doing using namespace foo, and some
other source forgetting the same because, in the unified case, they
benefit from the other source doing it. I think in the light of unified
sources, we should forbid non-scoped use of using.
That is:
using namespace foo;
would be forbidden, but
namespace bar {
using namespace foo;
}
wouldn't. In most cases, bar could be mozilla anyways.
Thoughts?
Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Thinking about the merge with unified build
Hi, Unified builds have been around for about two weeks. They are nice because they make the build so much faster. On the other hand, we know they currently break crash reports on mac (bug 943695), and can break things in subtle ways (bug 942421, bug 943839). We don't know what problems are still hiding. I think it's time, 9 days before the merge, to think about whether we want unified builds to ride the train or not. I'm almost tempted to suggest that we disable unified builds on nightlies, but that would probably hide the problems even more. Incidentally, in those two weeks, I did two attempts at building without unified sources, resulting in me filing 4 bugs in different modules for problems caused by 6 different landings[1]. I think it is time to seriously think about having regular non-unified builds (bug 942167). If that helps, I can do that on birch until that bug is fixed. Thoughts? Mike 1. For the curious: bugs 942137, 944830, 944834 and 944835, caused by bugs 932102, 942492, 943293, 944172, 943712 and 941559. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Thinking about the merge with unified build
On Sat, Nov 30, 2013 at 12:39:59PM +0900, Mike Hommey wrote: Hi, Unified builds have been around for about two weeks. They are nice because they make the build so much faster. On the other hand, we know they currently break crash reports on mac (bug 943695), and can break things in subtle ways (bug 942421, bug 943839). We don't know what problems are still hiding. I think it's time, 9 days before the merge, to think about whether we want unified builds to ride the train or not. I'm almost tempted to suggest that we disable unified builds on nightlies, but that would probably hide the problems even more. Incidentally, in those two weeks, I did two attempts at building without unified sources, resulting in me filing 4 bugs in different modules for problems caused by 6 different landings[1]. I think it is time to seriously think about having regular non-unified builds (bug 942167). If that helps, I can do that on birch until that bug is fixed. To add to the fun, I just tried to reduce the number of unified sources from 16 to 8, to see how build times would be affected. Guess what? That fails to build too. Filed bug 944844. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Mitigating unified build side effects Was: Thinking about the merge with unified build
If everyone puts using namespace foo; inside namespace mozilla {}, that
won't help much, right?
I'd prefer to minimize the source code changes required here. A tinderbox
non-unified build seems like the way to go.
Rob
--
Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa
stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr,
'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp
waanndt wyeonut thoo mken.o w
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Mitigating unified build side effects Was: Thinking about the merge with unified build
On Fri, Nov 29, 2013 at 8:01 PM, Mike Hommey m...@glandium.org wrote:
That is:
using namespace foo;
would be forbidden, but
namespace bar {
using namespace foo;
}
wouldn't. In most cases, bar could be mozilla anyways.
Most XPConnect stuff isn't in an explicit namespace, but the convention is
for the .cpp files to |using namespace JS;| - so this wouldn't work there,
nor is it a problem unless we start unifying source files across
directories and modules.
bholley
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
