Re: Proposal for an inbound2 branch
Ryan VanderMeulen wrote: My proposal: -Create an inbound2 branch identically configured to mozilla-inbound. -Under normal circumstances (i.e. m-i open), inbound2 will be CLOSED. -In the event of a long tree closure, the last green changeset from m-i will be merged to inbound2 and inbound2 will be opened for checkins. ---It will be a judgment call for sheriffs as to how long of a closure will suffice for opening inbound2. -When the bustage on m-i is resolved and it is again passing tests, inbound2 will be closed again. -When all pending jobs on inbound2 are completed, it will be merged to m-i. Why do we need a separate repo? Can we not simply close the broken head and start again at the last green changeset? -- Warning: May contain traces of nuts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal for an inbound2 branch
On 2013-05-02 5:46 AM, Neil wrote: Ryan VanderMeulen wrote: My proposal: -Create an inbound2 branch identically configured to mozilla-inbound. -Under normal circumstances (i.e. m-i open), inbound2 will be CLOSED. -In the event of a long tree closure, the last green changeset from m-i will be merged to inbound2 and inbound2 will be opened for checkins. ---It will be a judgment call for sheriffs as to how long of a closure will suffice for opening inbound2. -When the bustage on m-i is resolved and it is again passing tests, inbound2 will be closed again. -When all pending jobs on inbound2 are completed, it will be merged to m-i. Why do we need a separate repo? Can we not simply close the broken head and start again at the last green changeset? Multiheaded repos are evil. They're very hard to deal with, and I don't think that buildbot can deal with them properly. Also, they cannot be represented in git, which means that they will break the git mirror. Cheers, Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal for an inbound2 branch
Ehsan Akhgari wrote: Multiheaded repos are evil. They're very hard to deal with, and I don't think that buildbot can deal with them properly. Also, they cannot be represented in git, which means that they will break the git mirror. Why does mozilla-inbound need a git mirror? -- Warning: May contain traces of nuts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal for an inbound2 branch
On Thu, May 02, 2013 at 09:21:57AM -0400, Ehsan Akhgari wrote: Also, they cannot be represented in git I doubt this is true. The bridging tools may well not support it, but there's nothing structural about multiple-head repository that would prevent them to be mirrored in git. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal for an inbound2 branch
On 5/2/2013 6:21 AM, Ehsan Akhgari wrote: On 2013-05-02 5:46 AM, Neil wrote: Why do we need a separate repo? Can we not simply close the broken head and start again at the last green changeset? Multiheaded repos are evil. They're very hard to deal with, and I don't think that buildbot can deal with them properly. Buildbot can deal with a multi-headed Try, so it can probably be convinced to deal with a multi-headed inbound, even if it needs to be a special case like Try. Individual developers might have more problems (they'll want to pull from inbound, unlike Try) but I think as long as the inactive heads are closed it won't freak out Mercurial too much... The suggested workflow with inbound2 would lead to almost-identical multi-headed repos in developers' local clones anyway. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal for an inbound2 branch
On 5/2/13 10:09 AM, Matt Brubeck wrote: The suggested workflow with inbound2 would lead to almost-identical multi-headed repos in developers' local clones anyway. Only temporary ones, until inbound and inbound2 merge. -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
smartmake-like functionality has landed in mach
Hello dev-platform, If you don't use mach, this message does not concern you. If you use mach, in particular |mach build DIRECTORY|, keep reading. Bug 677452 landed smartmake-like functionality into mach, which I have dubbed dumbmake. smartmake (and dumbmake) [1] maintains a list of dependencies between source directories that are not captured in our recursive make configuration [2]. This means that |mach build DIRECTORY| might actually build several directories. This change does not affect |mach build|, and you can disable the new functionality on the command line with |mach build -X|. This is a short term step towards making |mach build| create functional builds more often, while requiring developers to internalize fewer dependencies. Long term, any dependency information added by Bug 677452 should be captured by the ongoing efforts to Build Faster [3]. I would appreciate help improving the list of dependencies. The list is maintained at build/dumbmake-dependencies; see python/mozbuild/dumbmake/README.rst for a description of the format. Finally, I'd like to thank Josh Matthews, the original author of smartmake, and Till Schneidereit, who added significant functionality and provided valuable guidance on how to integrate some of these ideas into mach. Yours, Nick Alexander [1] http://hg.mozilla.org/users/josh_joshmatthews.net/smartmake [2] smartmake can also maintain timestamps and deduce what directories need to be built, but this isn't implemented in mach. [3] Harder, better, ..., stronger. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Mac OSX 10.7+ lion style scroll bars landed on m-i
Hi, This is a quick heads up that Mac OSX 10.7+ lion style scroll bars (bug 636564) have landed on mozilla-inbound with https://hg.mozilla.org/integration/mozilla-inbound/rev/0c513ba74137. Once merged to mozilla-central expect them to appear in our nightlies. If you have any feedback or run into problems, please feel free to reach out to me or, even better, file bugs and CC me on them. If you have a need to revert to the old style scroll bars, here are two options: 1. Under System Preferences General, select Show scroll bars: Always. 2. Change nsLookAndFeel::UseOverlayScrollbars to always return false. Thanks, Stephen ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Mac OSX 10.7+ lion style scroll bars landed on m-i
Hooray! Thanks for all the hard work, Stephen. Rob -- q“qIqfq qyqoquq qlqoqvqeq qtqhqoqsqeq qwqhqoq qlqoqvqeq qyqoquq,q qwqhqaqtq qcqrqeqdqiqtq qiqsq qtqhqaqtq qtqoq qyqoquq?q qEqvqeqnq qsqiqnqnqeqrqsq qlqoqvqeq qtqhqoqsqeq qwqhqoq qlqoqvqeq qtqhqeqmq.q qAqnqdq qiqfq qyqoquq qdqoq qgqoqoqdq qtqoq qtqhqoqsqeq qwqhqoq qaqrqeq qgqoqoqdq qtqoq qyqoquq,q qwqhqaqtq qcqrqeqdqiqtq qiqsq qtqhqaqtq qtqoq qyqoquq?q qEqvqeqnq qsqiqnqnqeqrqsq qdqoq qtqhqaqtq.q ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: smartmake-like functionality has landed in mach
On 13-05-02 3:09 PM, Josh Matthews wrote: According to http://mxr.mozilla.org/mozilla-central/source/build/dumbmake-dependencies#8, it is equivalent to the following: ./mach build -X chrome xpcom toolkit/library That's correct. Or, if you're not a mach user, it translates to make -f client.mk chrome make -f client.mk xpcom make -f client.mk toolkit/library Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Storage in Gecko
- Original Message - Great post, Taras! Per IRC conversations, we'd like to move subsequent discussion of actions into a meeting so we can more quickly arrive at a resolution. Please meet in Gregory Szorc's Vidyo Room at 1400 PDT Tuesday, April 30. That's 2200 UTC. Apologies to the European and east coast crowds. If you'll miss it because it's too late, let me know and I'll consider moving it. https://v.mozilla.com/flex.html?roomdirect.htmlkey=yJWrGKmbSi6S Did someone post a summary of this meeting? Is there a link to share? Lawrence On 4/29/13 10:51 AM, Taras Glek wrote: * How to robustly write/update small datasets? #3 above is it for small datasets. The correct way to do this is to write blobs of JSON to disk. End of discussion. Writes of data = ~64K should just be implemented as atomic whole-file read/write operations. Those are almost always single blocks on disk. Writing a whole file at once eliminates risk of data corruption. Incremental updates are what makes sqlite do the WAL/fsync/etc dance that causes much of the slowness. We invested a year worth of engineering effort into a pure-js IO library to facilitate efficient application-level IO. See OS.File docs, eg https://developer.mozilla.org/en-US/docs/JavaScript_OS.File/OS.File_for_the_main_thread As you can see from above examples, manual IO is not scary If one is into convenience APIs, one can create arbitrary json-storage abstractions in ~10lines of code. * What about writes 64K? Compression gives you 5-10x reduction of json. https://bugzilla.mozilla.org/show_bug.cgi?id=846410 Compression also means that your read-throughput is up to 5x better too. * What about fsync-less writes? Many log-type performance-sensitive data-storage operations are ok with lossy appends. By lossy I mean data will be lost if there is a power outage within a few seconds/minutes of write, consistency is still important. For this one should create a directory and write out log entries as checksummed individual files...but one should really use compression(and get checksums for free). https://bugzilla.mozilla.org/show_bug.cgi?id=846410 is about facilitating such an API. Use-cases here: telemetry saved-sessions, FHR session-statistics. * What about large datasets? These should be decided on a case-by-case basis. Universal solutions will always perform poorly in some dimension. * What about indexeddb? IDB is overkill for simple storage needs. It is a restrictive wrapper over an SQLite schema. Perhaps some large dataset (eg an addressbook) is a good fit for it. IDB supports filehandles to do raw IO, but that still requires sqlite to bootstrap, doesn't support compression, etc. IDB also makes sense as a transitional API for web due to the need to move away from DOM Local Storage... * Why isn't there a convenience API for all of the above recommendations? Because speculatively landing APIs that anticipate future consumers is risky, results in over-engineering and unpleasant surprises...So give us use-cases and we(ie Yoric) will make them efficient. Taras ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: smartmake-like functionality has landed in mach
On 5/2/2013 3:45 PM, Nick Alexander wrote: On 13-05-02 3:09 PM, Josh Matthews wrote: According to http://mxr.mozilla.org/mozilla-central/source/build/dumbmake-dependencies#8, it is equivalent to the following: ./mach build -X chrome xpcom toolkit/library That's correct. Or, if you're not a mach user, it translates to make -f client.mk chrome make -f client.mk xpcom make -f client.mk toolkit/library Nick Does it also build browser/app on OSX after every build? Since that is pretty much required all the time and often missed. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Storage in Gecko
On 5/2/2013 4:13 PM, Lawrence Mandel wrote: - Original Message - Great post, Taras! Per IRC conversations, we'd like to move subsequent discussion of actions into a meeting so we can more quickly arrive at a resolution. Please meet in Gregory Szorc's Vidyo Room at 1400 PDT Tuesday, April 30. That's 2200 UTC. Apologies to the European and east coast crowds. If you'll miss it because it's too late, let me know and I'll consider moving it. https://v.mozilla.com/flex.html?roomdirect.htmlkey=yJWrGKmbSi6S Did someone post a summary of this meeting? Is there a link to share? Notes at https://etherpad.mozilla.org/storage-in-gecko We seemed to converge on a (presumably C++-based) storage service that has named branches/buckets with specific consistency, flushing, etc guarantees. Clients would obtain a handle on a branch, and perform basic I/O operations, including transactions. Branches could be created ad-hoc at run-time. So add-ons could obtain their own storage namespace with the storage guarantees of their choosing. Under the hood storage would be isolated so failures in one component wouldn't affect everybody. We didn't have enough time to get into prototyping or figuring out who would implement it. Going forward, I'm not sure who should own this initiative on a technical level. In classical Mozilla fashion the person who brings it up is responsible. That would be me. However, I haven't written a single line of C++ for Firefox and I have serious doubts I'd be effective. Perhaps we should talk about it at the next Platform meeting. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: smartmake-like functionality has landed in mach
On 05/03/2013 12:35 AM, Nick Alexander wrote: On 13-05-02 4:23 PM, Dave Townsend wrote: On 5/2/2013 3:45 PM, Nick Alexander wrote: On 13-05-02 3:09 PM, Josh Matthews wrote: According to http://mxr.mozilla.org/mozilla-central/source/build/dumbmake-dependencies#8, it is equivalent to the following: ./mach build -X chrome xpcom toolkit/library That's correct. Or, if you're not a mach user, it translates to make -f client.mk chrome make -f client.mk xpcom make -f client.mk toolkit/library Nick Does it also build browser/app on OSX after every build? Since that is pretty much required all the time and often missed. Looking at http://mxr.mozilla.org/mozilla-central/source/build/dumbmake-dependencies, it does not. I considered supporting conditional dependencies like this, but was torn between using the preprocessor or using the expression parser used for test manifests. I decided to land the simplest thing that could possibly work first, but I'm game for improving the format to support conditionals. Yours, Nick Given that browser/app is a pretty quick build (afaik), we could probably just throw it into the list as a global dependency. Cheers, Josh ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Storage in Gecko
Whatever you do, please, please, please make sure that everything is worker-friendly. If we can't write (or at least read) contents to that Key-Value store from a worker, we will need to reimplement everything in a few months. Cheers, David - Original Message - From: Gregory Szorc g...@mozilla.com To: Lawrence Mandel lman...@mozilla.com Cc: David Rajchenbach-Teller dtel...@mozilla.com, Taras Glek tg...@mozilla.com, dev-platform dev-platform@lists.mozilla.org Sent: Friday, May 3, 2013 1:36:15 AM Subject: Re: Storage in Gecko On 5/2/2013 4:13 PM, Lawrence Mandel wrote: - Original Message - Great post, Taras! Per IRC conversations, we'd like to move subsequent discussion of actions into a meeting so we can more quickly arrive at a resolution. Please meet in Gregory Szorc's Vidyo Room at 1400 PDT Tuesday, April 30. That's 2200 UTC. Apologies to the European and east coast crowds. If you'll miss it because it's too late, let me know and I'll consider moving it. https://v.mozilla.com/flex.html?roomdirect.htmlkey=yJWrGKmbSi6S Did someone post a summary of this meeting? Is there a link to share? Notes at https://etherpad.mozilla.org/storage-in-gecko We seemed to converge on a (presumably C++-based) storage service that has named branches/buckets with specific consistency, flushing, etc guarantees. Clients would obtain a handle on a branch, and perform basic I/O operations, including transactions. Branches could be created ad-hoc at run-time. So add-ons could obtain their own storage namespace with the storage guarantees of their choosing. Under the hood storage would be isolated so failures in one component wouldn't affect everybody. We didn't have enough time to get into prototyping or figuring out who would implement it. Going forward, I'm not sure who should own this initiative on a technical level. In classical Mozilla fashion the person who brings it up is responsible. That would be me. However, I haven't written a single line of C++ for Firefox and I have serious doubts I'd be effective. Perhaps we should talk about it at the next Platform meeting. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
