Re: Using MozPhab without Arcanist
Am 24.10.19 um 12:13 schrieb Henrik Skupin: glob wrote on 23.10.19 17:56: It's available now - make sure you're running the latest version by running `moz-phab self-update`. That's what I did yesterday, but as it looks like the self-update actually didn't update my version to the latest MozPhab-0.1.55. I will check soon with this version. Thanks! Henrik You need to run self-update twice to move over to the pip version. Also, make sure to not run it while in a virtualenv like I did. Otherwise you end up uninstalling and installing it from scratch ;-) Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
mach lint -l l10n coming your way
Hi, I just landed a new linter into mach, and thus into treeherder and phabricator. It's called `l10n`, and `l1nt` on treeherder. It checks for common errors in localizable files, like duplicate strings and parsing errors, but also runs some more detailed checks. On phabricator, it also reports warnings, which might be missed string ID changes. So if a patch doesn't show the strings you're hoping for, a good first step is to run ./mach lint -w now. Together with l10n jobs on try and pseudo localization [1], this should make your life easier. On string changes: The rules remain the same, if you change the semantics of a string, you need a new ID. If you just fix a typo or grammar, please don't create a new ID. Anything inbetween, feel free to ask, often the best choice depends on what's actually in the localizations. Technical detail: The test works by maintaining a local clone of https://hg.mozilla.org/l10n/gecko-strings/ in your ~/.mozbuild, updating it after two days. So if you see something cloning while you mach lint, that's intended. This comes with the benefit that we're only checking against strings we actually expose to localizers. It also checks for string IDs we might not use in central, but in beta or release. Which also makes this test the right thing to check for landings on beta or release branches. Thanks to Andrew Halberstadt for bearing with me through all the refactors while he was trying to help me land this. There are a few next steps I already know about: For one, run this more widely across mozilla projects, for example on our newer Android apps [2]. I also have a local branch for ' vs ’, but it's still in its infancy. If you find issues and/or want enhancements, please file a bug in linting and CC me. Axel [1] https://firefox-source-docs.mozilla.org/intl/l10n/l10n/fluent_tutorial.html#pseudolocalization [2] https://bugzilla.mozilla.org/show_bug.cgi?id=1548500 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Patching FF 52.9esr with v60+ Security updates possible?
Disclaimer, I'm not a security expert, but a couple of thoughts: We have rewritten 52.x code in Rust, and we have removed features. If there are security vulnerabilities in the 52.x versions of that code, nobody is going to tell Mozilla. In that sense, it's unlikely that Mozilla will ever have the list of known vulnerabilities in your patched 52.x code base. Some security fixes we made might be executed by writing components in Rust. Just assuming that Rust and the modern compiler toolchain it requires is part of your problem, you won't be able to port these fixes. My personal take is that you may be able to apply a lot of the patches that have CVEs, but that's likely not going to get you a code base that is similarily secure as the one we're working on. Axel Am 13.04.19 um 00:43 schrieb Charles Robertson: Hi, I know this sounds like a strange questions. However, we have a very large customer who is using our old OS which the last successful build of Firefox ESR was 52.9. But because of the massive updates to FF 60 we have been unable to get FF 60+ to build on that old OS. This customer has demanded we provide an updated Firefox for this old OS so I am asking if it would be possible to patch FF 52.9esr with the security updates since 60 was released? Thanks, Cheers Charles Robertson Firefox Maintainer - SUSE ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Improving our usage of Bugzilla
Am 02.04.19 um 19:24 schrieb Sylvestre Ledru: Because I had a few discussions about task vs enhancement, a good way to make the difference between the two use cases is: If I ever need help with this bug, should it come from someone in Product or an EPM? Can I ask for clarification? Is it task - EPM help enhancement - Product help or the other way around? As you listed them the other way around, and that's not how I'd line them up. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Min clang / libclang requirement was updated not long ago...
Am 27.02.19 um 15:28 schrieb Nathan Froyd: On Wed, Feb 27, 2019 at 9:05 AM Axel Hecht wrote: Am 27.02.19 um 14:39 schrieb Nathan Froyd: On Wed, Feb 27, 2019 at 6:22 AM Kartikaya Gupta wrote: On Wed, Feb 27, 2019 at 3:40 AM Axel Hecht wrote: Can we please not force bootstrap? +1. In general bootstrap isn't "rock solid" enough to force people into running it. If people have problems with bootstrap (it doesn't do enough, it assumes too much about your system, etc. etc.), please file bugs on what's wrong. We need to start depending more on bootstrap for everything, to the point of "you can't depend on X unless it gets installed via bootstrap", and we can't get to that world if we don't know what rough edges people find in bootstrap. Do you have a suggestion on how to do that in practice? Rolling back from a broken development environment is easily a couple of hours of work, in the case of homebrew breaking all my virtualenvs, for example. It's not clear to me what bootstrap does that breaks things. Do you want the ability to skip installing everything via homebrew? The list of things that I consider broken in homebrew is dynamic, and I guess not worth enumerating. I have installed hg outside of homebrew, and using a homebrew python for any virtualenv can break at any invocation of homebrew, and as such also for building firefox. Any invocation because homebrew recently decided to not just auto-update, but also to auto-cleanup. See ls -al obj-firefox-repack/_virtualenvs/init/ total 16 drwxr-xr-x 8 axelhecht wheel 256 Feb 21 11:58 . drwxr-xr-x 3 axelhecht wheel 96 Feb 21 11:58 .. lrwxr-xr-x 1 axelhecht wheel 83 Feb 21 11:58 .Python -> /usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/Python drwxr-xr-x 21 axelhecht wheel 672 Feb 21 11:58 bin drwxr-xr-x 3 axelhecht wheel 96 Feb 21 11:58 include drwxr-xr-x 3 axelhecht wheel 96 Feb 21 11:58 lib -rw-r--r-- 1 axelhecht wheel 61 Feb 21 11:58 pip-selfcheck.json -rw-r--r-- 1 axelhecht wheel 15 Feb 21 11:58 python_exe.txt and that symlink disappears for each upgrade of python. And the way that bootstrap interacts with homebrew doesn't allow to pin revisions, or at least it broke at some point. To reiterate, I don't intend to run bootstrap to figure out what damage it does. It might cost me a day, and fills me up with anger that's not good for anybody. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Min clang / libclang requirement was updated not long ago...
Am 27.02.19 um 14:39 schrieb Nathan Froyd: On Wed, Feb 27, 2019 at 6:22 AM Kartikaya Gupta wrote: On Wed, Feb 27, 2019 at 3:40 AM Axel Hecht wrote: Can we please not force bootstrap? +1. In general bootstrap isn't "rock solid" enough to force people into running it. If people have problems with bootstrap (it doesn't do enough, it assumes too much about your system, etc. etc.), please file bugs on what's wrong. We need to start depending more on bootstrap for everything, to the point of "you can't depend on X unless it gets installed via bootstrap", and we can't get to that world if we don't know what rough edges people find in bootstrap. Do you have a suggestion on how to do that in practice? Rolling back from a broken development environment is easily a couple of hours of work, in the case of homebrew breaking all my virtualenvs, for example. Axel Thanks, -Nathan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Min clang / libclang requirement was updated not long ago...
Can we please not force bootstrap? Background: As a python-heavy engineer, my relationship with homebrew is pretty broken these days. There's stuff that's essential to my day-to-day business that I had to take out of homebrew, including having my own compiles of python. I also have to have my own install of hg, for the sake of extensions I use. ./mach bootstrap interferes with those decisions, so I can't run it anymore. Axel Am 26.02.19 um 19:17 schrieb Chris Peterson: Seems like mach bootstrap should have a clobber flag so anyone updating build tool dependencies can force people to re-run mach bootstrap (and save many people frustration). On 2/26/2019 10:00 AM, David Major wrote: Does configure warn about this? The link between this error and needing to bootstrap is not super clear (and a surprising number of people don't read dev-platform) so I'm not looking forward to answering the same question in #build for the rest of the week. :) On Tue, Feb 26, 2019 at 12:23 PM Emilio Cobos Álvarez wrote: ... so if you don't use the mozilla-provided libclang (or are using a very old one), and you see an error like: error[E0277]: the trait bound `values::generics::rect::Rect f32>>: std::convert::From, Please re-run mach bootstrap, or update your libclang. Thanks! -- Emilio ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: New and improved "about:config" for Firefox Desktop
Is there a tracking bug for follow-ups? I'd have a few, adding pref w/out search (*), show add on screen for long searches, filter/order by modified, search in values, can't abort edit. (*) I just realize that I didn't understand how "add" works. Maybe the bug is to make that discoverable? Axel Am 24.01.19 um 20:31 schrieb Paolo Amadini: Last year a group of students, Luke, Matthias, and Vincent, designed and implemented a new version of "about:config" in order to improve the ergonomics and align the look and feel with other in-content Firefox pages. They did an amazing job, working with design input from Amy Lee and with myself doing code reviews. I'm happy to announce that this work will be available to everyone in Firefox 67, and can be already used in Nightly at this URL: chrome://browser/content/aboutconfig/aboutconfig.html Most improvements are the natural result of using HTML instead of XUL: * There are visible buttons for editing preferences * String values are displayed in full as multiline text * Find in page works for both names and values * Triple click selects one preference name or value quickly * Text selection works on multiple preferences * The context menu is the same as regular web pages - Copy to the clipboard - Open selected link - Search with your preferred engine * Search results don't include spurious value matches anymore * Closing and reopening the browser while the tab is pinned preserves the search term We've not just converted the old page, we've designed something new based on actual use cases, telemetry data, and opportunity cost. We preferred natural HTML page interactions, for example a double click now selects text instead of toggling the value. The way the page is explored with screen readers has also changed, and we've ensured that the new way is still clear and easy to use. We're still keeping the old "about:config" around at the following URL for a while, to mitigate risk related to unforeseen circumstances: chrome://global/content/config.xul Thunderbird will not be affected by this change initially, but at some point we'll remove the old code from mozilla-central since Thunderbird will be the only remaining user. *Performance* This page can be slower than the old one in some cases. On slower machines the page may take a moment to display all preferences, if you so choose. We worked around this by waiting for the first input before displaying results, as 93% of "about:config" page shows include a search anyway. Navigation, scrolling, and find in page are then fast. We've used performance profiling to optimize the page and avoid the slowest layout modes, but we've not compromised on using the latest best practices for Firefox Desktop like Fluent localization, which are anyways in the process of being optimized on their own. We've explicitly chosen to avoid virtualizing the list, that is only rendering visible DOM nodes, because this would add complexity that is not needed for an internal page. It would also nullify most of the advantages in accessibility and usability that we gained at a low cost just because we're using a simple HTML table. Effort would be better spent on optimizing the web for the layout of tables of about 3,000 rows, which would benefit every web site instead of Firefox only. *Tutorials and screenshots on the web* While with some features there is a concern that a change would make it more difficult for users to follow instructions found in older tutorials on the web, this is much less of a concern in this case, given that the page caters to experienced users and the changes affect presentation rather than actual functionality. In fact, existing information on the web can more easily become obsolete because the preferences go away or change their meaning, rather than because of a change in how the values can be changed. *Features that have not been rewritten* If the new page is missing a feature that the old one used to have, there is probably a good reason. Luke added telemetry probes to the current "about:config" so we know how people use it. It's basically just one mode of operation across all channels: search, then maybe edit or add a preference. There are more details in the history section below, but this is to say that it is unlikely that we would accept a patch to add back a certain feature just because it used to be present before. All patches would have to be motivated by an actual need and include exhaustive regression tests. That said, we have ideas for supporting new use cases for browser developers, like pinning a list of favorites or just showing recently modified preferences first, but we don't plan to start working on them before the current version reaches Release. *More details on history, motivation, and process* If you're reading this you probably already have a good idea of what we're talking
Re: C++ standards proposal for a embedding library
CCing snorp. I guess it's interesting to see how the geckoview API differs from the webview API, and which of those differences are related to goal of that C++ API, and which are more browser-focused. And if the C++ API should be also browser-focused, in the end. Not making any statement on the interesting question of such a stdlib thing, and how that impacts choice and innovation on the web space. Which should be a more important question probably for mozilla. Axel Am 18.07.18 um 21:55 schrieb Botond Ballo: On Wed, Jul 18, 2018 at 3:32 PM, Jeff Gilbert wrote: It feels like the committee is burnt out on trying to solve the general library problem, but contemplating something massively complex like this instead doesn't follow, and is an answer to the wrong question. Make it easier to integrate libraries and we wouldn't see kludge proposals like this. Could you elaborate on the "complexity" and "kludge" aspects? One of the main complaints about the 2D graphics proposal was that it was trying to create a new spec in a space where there are existing mature specs, and that the committee as a group doesn't necessarily have the depth of domain expertise in graphics necessary to specify a library like this. This web_view proposal attempts to address that concern by leveraging existing graphics and other specs from web standards. So, in a sense, the committee is trying to avoid dealing with complexity / reuse the work that others have done to tackle the complexity inherent in the problem space. If you're referring to the embedding mechanism / API itself being complex, it would be useful to elaborate on why. The API surface in the proposed library seems to be quite small. It's also worth noting that there is prior art in this space in the form of e.g. the QtWebView and wxWebView APIs, which I believe are fairly popular in cross-platform C++ applications, suggesting a demand for this sort of library. Note that I'm not necessarily advocating for this proposal; I'm just trying to understand the concerns / feedback better so that I can communicate them to the proposal authors effectively. If you would prefer to communicate the concerns to the authors directly, please feel free to do so. Thanks, Botond ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent To Require Manifests For Vendored Code In mozilla-central
A couple of comments: One thing I'm missing is the ability to do mono-repo imports. Say we want to vendor in https://github.com/projectfluent/fluent.js/tree/master/fluent-gecko. For js libraries, we might also want to pay attention to .npmignore (others already mentioned hg, so also .hgignore). There's no spec what happens with patches that fail to apply, or failed run_after scripts. Do we intend to do something if the LICENSE changes? Also, what are we supposed to do if the vendored code doesn't have a LICENSE file? Axel Am 10.04.18 um 06:25 schrieb glob: mozilla-central contains code vendored from external sources. Currently there is no standard way to document and update this code. In order to facilitate automation around auditing, vendoring, and linting we intend to require all vendored code to be annotated with an in-tree YAML file, and for the vendoring process to be standardised and automated. The plan is to create a YAML file for each library containing metadata such as the homepage url, vendored version, bugzilla component, etc. See https://goo.gl/QZyz4xfor the full specification. We will work with teams to add moz.yaml files where required, as well as adding the capability for push-button vendoring of new revisions. Please address comments to the dev-platform list. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: FYI: Short Nightly Shield Study involving DNS over HTTPs (DoH)
I have a couple of further questions: One is about the legal impact on users. DNS mangling is part of law enforcement strategies in many parts of the world (incl Germany). We should restrict this experiment to regions where Mozilla knows that there's no legal trouble of using DoH and cloudflare. Circumventing law enforcement can get pretty hairy in some regions, I suspect. The other is a bit more detail on the scope of Mozilla's agreement with cloudflare extending the experiment. Does our agreement extend to people not using Firefox? What happens to folks that in some weird way are stuck with the experiment DoH setup once the experiment ends? It'd be a great pitch if the agreement was that cloudflare offers this service with said terms. If they stopped liking the terms, they'd have to shut the service down. These questions are really only about the scope, not so much about if we should do it. Axel Am 19.03.18 um 18:08 schrieb Selena Deckelmann: Hi! Thanks for all the thoughtful comments about this experiment. The intent of this work is to provide users privacy-respecting DNS. Status quo for DNS does not offer many users reasonable, informed choice about log retention, and doesn't offer encrypted DNS. Users also cannot be reasonably expected to negotiate on their own with their ISPs/VPN providers for things like 24-hour retention for logs that can be used to create profiles. Today's default environment (speaking technically wrt lack of encryption and log storage, and also in terms of the regulatory environment in the US) allows *all* of this data to be collected indefinitely and sold to third parties. There's a lot of thinking that went into the agreement we have with Cloudflare to enable this experiment in a way that respects user privacy. We also want to explain the impact we think this kind work will have on the privacy of the Internet. I'd like the team to share this in a blog post about the experiment, and so have started work with them on it. More on this shortly! -selena On Mon, Mar 19, 2018 at 8:16 AM Daniel Stenberg <dstenb...@mozilla.com> wrote: On Mon, 19 Mar 2018, Martin Thomson wrote: I don't know if it is possible to know if you have a manually-configured DNS server, but disabling this experiment there if we can determine that would be good - that might not be something to worry about with Nightly, but it seems like it might be needed for this to hit the trains. How do we otherwise determine that a DNS server is not safe to replace? Split horizon DNS is going to cause unexpected failures when users - particularly enterprise users - try to reach names that aren't public. That's not just an enterprise thing; this will break my home router in some ways as well, though I'm actually totally OK with that in this case :) I don't think it is possible - with any particularly high degree of certainty - to know if a DNS server has been manually configured (or even if the term itself is easy to define). The system APIs for name lookups typically don't even expose which DNS server they use, they just resolve host names to addresses for us. For TRR, we've instead focused pretty hard on providing a "retry-algorithm" so that Firefox can (if asked), retry a failed name resolve or TCP connect without TRR and then "blacklist" that host for further TRR use for a period into the future. For hosts that are TRR-blacklisted this way, we also check the next-level domain of it in the background to see if we should also blacklist the whole domain from TRR use. Ie if "www.example.com" fails with TRR, it gets blacklisted, retried with the native resolver and "example.com" is tested to see if the entire domain should be blacklisted. -- / daniel.haxx.se ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Password autofilling
Am 02.01.18 um 17:22 schrieb Gijs Kruitbosch: On 01/01/2018 20:08, Jonathan Kingston wrote: We have the ability to turn off the whole login manager within Firefox preferences: "Remember logins and passwords for web sites" but no way to prevent autofill. There's an about:config pref, as [1] points out, which does this. I wonder if there's a way to require user interaction only when pages contain non-same-origin scripts. Then again, it's not clear that that'd be "worth it", in the sense that that would actually significantly reduce the number of pages where user interaction would be required, nor that it wouldn't make the browser's behaviour less understandable to end users (as we would sometimes autofill without interaction, and sometimes wouldn't). In other form code we also care about whether form fields are focusable (ie visible, editable etc.), which is something we could also potentially use to mitigate these attacks, though it could probably be bypassed by having a visible element that is positioned "offscreen" in an overflow:hidden container, or something of that sort. ~ Gijs Or could we start blocking tracking-providers with this practice in general? As much as this sounds like an arm-race, these providers are only valuable if they're on a lot of sites, so this might actually be a winnable arm-race. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: INTENT TO DEPRECATE (taskcluster l10n routing)
Am 04.12.17 um 05:42 schrieb Jet Villegas: On Sun, Dec 3, 2017 at 05:15 Axel Hecht <l...@mozilla.com <mailto:l...@mozilla.com>> wrote: Am 01.12.17 um 16:45 schrieb Justin Wood: > Hey Everyone, > > tl;dr if you don't download nightly l10n repacks via taskcluster index > routes, this does not affect you. > > Up until recently you could only find nightly l10n repacks with the > following routes: > > * > .gecko.v2.{project}.revision.{head_rev}.{build_product}-l10n.{build_name}-{build_type}.{locale} > * > .gecko.v2.{project}.pushdate.{year}.{month}.{day}.{pushdate}.{build_product}-l10n.{build_name}-{build_type}.{locale} > * > {index}.gecko.v2.{project}.latest.{build_product}-l10n.{build_name}-{build_type}.{locale} > > Recently I have updated the routing to match that of regular Nightlies, > specifically one such route is: > > gecko.v2.mozilla-central.nightly.revision.a21f4e2ce5186e2dc9ee411b07e9348866b4ef30.firefox-l10n.linux64-opt That's followed by locale code, right? I found gecko.v2.mozilla-central.nightly.revision.de1f7a92e8726bdd365d4bbc5e65eaa369fbc20a.firefox-l10n.macosx64-opt.de <http://gecko.v2.mozilla-central.nightly.revision.de1f7a92e8726bdd365d4bbc5e65eaa369fbc20a.firefox-l10n.macosx64-opt.de> > This deprecation is in preparation of actually building l10n repacks on > (nearly) every code checkin, rather than just on nightlies. Does that mean that you're deprecating all but that route, or are there more? > Let me know if there are any questions or concerns. No concerns, just curiousity. We're not running any tests on localized builds at this point, right? I hope we can change that (testing on localized builds) with this proposed change. We’ve gotten reports that localized builds (and related usage; e.g., input method editors) cause A11y API activation, which triggers other bugs for us. My gut reaction is "that shouldn't happen", though, well, no idea what IMEs do. Do we have bugs tracking these? I'd love to be on CC on those. As for running tests, we have 100 localizations and 5 platforms, we'll need to be pretty conservative on which tests we run so that we don't blow up our budget by a factor of 500. Also, many of our tests actually hard-code en-US artifacts, like "ensure that the button on this dialog says "Save", and thus will fail when run on localized builds. I don't have a list, though. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: INTENT TO DEPRECATE (taskcluster l10n routing)
Am 01.12.17 um 16:45 schrieb Justin Wood: Hey Everyone, tl;dr if you don't download nightly l10n repacks via taskcluster index routes, this does not affect you. Up until recently you could only find nightly l10n repacks with the following routes: * .gecko.v2.{project}.revision.{head_rev}.{build_product}-l10n.{build_name}-{build_type}.{locale} * .gecko.v2.{project}.pushdate.{year}.{month}.{day}.{pushdate}.{build_product}-l10n.{build_name}-{build_type}.{locale} * {index}.gecko.v2.{project}.latest.{build_product}-l10n.{build_name}-{build_type}.{locale} Recently I have updated the routing to match that of regular Nightlies, specifically one such route is: gecko.v2.mozilla-central.nightly.revision.a21f4e2ce5186e2dc9ee411b07e9348866b4ef30.firefox-l10n.linux64-opt That's followed by locale code, right? I found gecko.v2.mozilla-central.nightly.revision.de1f7a92e8726bdd365d4bbc5e65eaa369fbc20a.firefox-l10n.macosx64-opt.de This deprecation is in preparation of actually building l10n repacks on (nearly) every code checkin, rather than just on nightlies. Does that mean that you're deprecating all but that route, or are there more? Let me know if there are any questions or concerns. No concerns, just curiousity. We're not running any tests on localized builds at this point, right? Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal to remove some preferences override support
Looping in mkaply explicitly, if that has impact on organizational deployments. Axel Am 02.11.17 um 00:41 schrieb Nicholas Nethercote: Greetings, In https://bugzilla.mozilla.org/show_bug.cgi?id=1413413 I am planning to remove a couple of things relating to preferences. 1) Remove the defaults/preferences directory support for extensions. This is a feature that was used by legacy extensions but is not used by WebExtensions. 2) Remove the "preferences" override directory in the user profile. This removes support for profile preferences override files other than user.js. The bug has a patch with r+. The specific things it removes include: - The "load-extension-default" notification. - The NS_EXT_PREFS_DEFAULTS_DIR_LIST/"ExtPrefDL" directory list, including the entry from the toolkit directory service. Does anybody foresee any problems with this change? Thanks. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Changes to tab min-width
Am 04.10.17 um 18:43 schrieb Jeff Griffiths: Om my system ( retina macbook pro ) 70 is starting to look like a better compromise for tab readability. How I have been testing this: - change the value to a specific number, say 70 - open enough tabs so that overflow triggers, then close two tabs, then open a tab ( we retain overflow until 2 tabs have been closed! ) - count the number of tabs opened - open chrome and open that number of tabs - compare the utility of each browser I tested 70 and 75 (which Aaron suggested), and so far 75 is OK, 70 is crossing the border to my tab claustrophobia. In particular on 50, I had trouble finding the right hit targets to select tabs or close them. And 70 still feels close to that, while 75 for me personally doesn't. I'll run with 75 for a couple more days. And yes, the profiles I'm trying this with are mostly tabs on similar sites, so the favicons don't provide any practical value. Axel Jeff On Wed, Oct 4, 2017 at 9:37 AM, Marco Bonardo <mbona...@mozilla.com> wrote: On Tue, Oct 3, 2017 at 10:36 PM, Jeff Griffiths <jgriffi...@mozilla.com> wrote: 1. do you prefer the existing behaviour or the new behaviour? 2. if you prefer a value for this pref different than 50 or 100, what is it? Why? I prefer being able to see a minimum part of the title, because I very often have multiple tabs open on the same page (many bugzilla, many searchfox, many crash-stats) and now I cannot distinguish them at all. But at the same time, I never liked much the scrolling behavior, at a point that when my tabs start scrolling, I begin a cleaning taks to close some of them. Looks like I'm unhappy in both cases, sorry. If I'd really have to pick, I'd probably would like to see the first 10 chars of the title. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Changes to how we localize Firefox - one repository to rule a hundred repositories
Hi, tl;dr: We'll be using a single French, German, Slovenian localization across all of mozilla-central, -beta, -release, -esr-*, starting with 57. This change will ride the trains. We call it "cross channel localization", or x-channel in short. How does that work? We're creating an artificial repository containing the strings from mozilla-central and comm-central. This will expand to contain also the strings on -beta, -release, etc, as the change rides the trains. This will be one repository, with a single head. See https://hg.mozilla.org/users/axel_mozilla.com/en-US for a current draft. We'll use our existing workflows to create localizations for that repository, and those will be hosted in https://hg.mozilla.org/l10n-central/. We'll use those repositories to just build as usual, from m-c, m-b, etc. Developer impact: We'll need string IDs to be unique across channels. But that's really it. We'll help with a test running in automation checking that, see https://bugzilla.mozilla.org/show_bug.cgi?id=1353680. Benefits for Firefox: We'll be shipping localizations quicker. We'll be shipping fixes to localizations quicker. We'll be dealing with string problems in a straight forward way. Overall, we're expecting localizers to be more involved, and more frequently involved, and to generally increase product quality by removing obstacles and confusing bits and pieces of where to fix a thing and how to make that fix stick. Are we done yet? No, there's going to be a number of things we'll need to improve after the initial landing. For one, we want to run the tool that creates the repository in automation. I'll run it at home for a while to tell the difference between problems in my code and your code. The other interesting bit is going to be to deal with closed heads and debugparents as part of merge days. But that has 6 weeks time, as the next couple of merges are only forks for x-channel for now. Timeline: We've been talking to various stakeholders in the Firefox team, to our VCS friends, to localizers for quite some time. Firefox 57 is a force in actually landing this, to deliver on our quality expecations. But also to deal with the slightly different merge plans that Sylvestre announced earlier today. Actually shipping x-channel with 57 is the only really sane and polite way to deal with that plan. The tracking bug is https://bugzilla.mozilla.org/show_bug.cgi?id=1353655. Happy to answer questions Axel PS: a hundred repositories to rule them all? Yeah, we're moving from 3-4 for each of our ~100 localizations to one for each. We intend to continue to ship a different version per locale and Firefox product going forward, and that's just easier to digest with independent repositories per locale. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Mac/Win32/Win64 Nightly builds for some locales aren't being updated since 08-08
Am 11.08.17 um 11:07 schrieb rodrigo.mcu...@hotmail.com: Can someone enlighten me on why some locales' Nightly builds are not being updated since August 8th? This probably isn't the best place to post this, but I don't know if this is intended or not so I rather ask here than submit a bug. Thanks! Please submit a bug if it's not working today, seems like https://bugzilla.mozilla.org/show_bug.cgi?id=1389260 got fixed for today's nightly. Also CC me? Thanks Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Phabricator and confidential reviews
To answer the question not asked ;-) I think we should strive to have as few people as possible with general access to security bugs. The concerns the folks have when crossing borders is awful. And just from a general risk profile. Saying that as someone that neither has nor wants access to security bugs in general. So, in that sense, I think we should make this a general assumption, that the folks writing patches and doing reviews are not in group of the bug. As to mirroring the information, I think it'd be good if there couldn't be people on phabricator that are not on the bug. That way, the folks on the bug that manage the security risk of it have one way of tracking the visibility of the security issue. I could see how not everybody involved in the bug automatically wants all the bugmail. I personally wouldn't mind if I could opt-in to that, but I'd be annoyed if I had to ask folks to manually add me again, after I asked them to CC me to the bug. More a question if it's technically feasible, could folks ask phabricator for access by following the link, and it would go back and check bugzilla on-demand if it should do that? Depends a bit on the additional private-attachment thing that Nicolas mentioned. Axel Am 09.08.17 um 02:30 schrieb Mark Côté: (Cross-posted to mozilla.tools) Hi, I have an update and a request for comments regarding Phabricator and confidential reviews. We've completed the functionality around limiting access to Differential revisions (i.e. code reviews) that are tied to confidential bugs. To recap the original plan, various security groups in BMO are mirrored to Phabricator as "projects", containing the same set of users. When a bug has such a security group added to it, e.g. dom-core-security, thus restricting its visibility largely to members of that group, a Phabricator "policy" is similarly set on any associated revisions, restricting their visibility to the same group of people (plus the author of the revision, if they are not in the project already). However, users outside of the security group(s) can see confidential bugs if they are involved with them in some way. Frequently the CC field is used as a way to include outsiders in a bug. Phabricator has a similar feature, called "subscribers", which, as with CCs, both grants visibility to confidential revisions and also sends email updates when the revision changes. It was suggested that we attempt to synchronize CC and subscriber lists. First I want to double check that this is truly useful. I am not sure how often CCed users are involved with confidential bugs' patches (I might be able to ballpark this with some Bugzilla searches, but I don't think it would be easy to get a straight answer). Anecdotally I have been told that a lot of the time users are CCed just to be informed of the problem, e.g. a manager might want to be aware of a vulnerability. Given that adding subscribers to a revision is just as easy as CCing a user on a bug, if it is infrequent that outsiders need to be involved in reviewing confidential patches, I lean towards taking the simple route of making this manual. However if this is more common than I suspect, then we must decide how to synchronize the lists. The most straightforward approach is one-way synchronization from BMO, that is, anyone CCed on the bug will automatically be added as a subscriber to any associated revisions, but anyone manually added to the subscribers list who is not CCed on the bug would be automatically removed by the BMO-Phabricator synchronization routine. The alternative is to keep track of who was manually added and who was automatically synchronized, which gets complicated rather quickly, both in terms of implementation and usability. The second question that would come up is whether this synchronization should apply to all revisions or just confidential ones. Given the dual nature of CCs/subscribers, for both visibility and notifications, I lean towards only doing this synchronization for confidential revisions, where it is more important. A further justification for limiting the mirroring is that Phabricator has a much more powerful and fine-grained notification system (Herald) than BMO's product- and component-watching feature. Automatic mirroring everywhere would reduce the utility of the former. If you have any thoughts on this, please reply. I'll answer any questions and summarize the feedback with a decision in a few days. Note that we can, of course, try a simple approach to start, and add in more complex functionality after an evaluation period. To sum up, there are three questions: 1. Is mirroring a confidential bug's CC list to association Differential revisions' subscriber lists actually useful? That is, does the utility justify the cost of implementation and maintenance? 2. If yes, is one-way mirroring, from BMO to Differential, sufficient?
Creating a localized build locally
Hi, cross-posting this from https://blog.mozilla.org/l10n/2017/08/04/create-a-localized-build-locally/. Yesterday we changed the way that you create localized builds on mozilla-central. This works for developers doing regular builds, as well as developers or localizers without a compile environment. Sadly, users of artifact builds are not supported [1]. For language packs, a mere ./mach build langpack-de will work. If you’d rather wish to build a localized package, you’ll want to get the package first. If you’re building yourself, that’s ./mach package and if you want to get a Nightly build from archive.mozilla.org, just ./mach build wget-en-US If you want to do that for Firefox for Android, you’ll need to specify which platform you want. Set EN_US_BINARY_URL to the latest-mozilla-central-* path for the binary you want to test. If you have a good suggestion for a default, we'd need something like https://hg.mozilla.org/mozilla-central/rev/64a69b2cebbb. And then you just ./mach build installers-fr That’ll take care about getting the french l10n repository, and do all the necessary things to get you a nice little installer/package in dist. Pick your favorite language from our repositories [2]. Care for a RTL build? ./mach installers-fa will get you a Persian build . As with other repositories we clone into ~/.mozbuild, you’ll want to update those every now and then. They’re in l10n-central/*, a repository for each language you tried. Either via hg or git-cinnabar, depending on your m-c checkout. Documentation is on https://gecko.readthedocs.io/en/latest/build/buildsystem/locales.html, bugs go to Core Build [3]. This works for Firefox, Firefox for Android, and Thunderbird. And now you can safely forget all the things you never wanted to know about localized builds. Axel [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1387485 [2] https://hg.mozilla.org/l10n-central/?sort=lastchange [3] https://bugzilla.mozilla.org/enter_bug.cgi?product=Core=Build%20Config=l...@mozilla.com ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: More Rust code
Am 17.07.17 um 21:43 schrieb Ted Mielczarek: Nick, Thanks for kicking off this discussion! I felt like a broken record talking to people about this in SF. From my perspective Rust is our single-biggest competitive advantage for shipping Firefox, and every time we choose C++ over Rust we throw that away. We know the costs of shipping complicated C++ code: countless hours of engineering time spent chasing down hard-to-reproduce crashes, exploitable security holes, and threading issues. Organizationally we need to get to a point where every engineer has the tools and training they need to make Rust their first choice for writing code that ships with Firefox. On Mon, Jul 10, 2017, at 09:15 PM, Bobby Holley wrote: I think this is pretty uncontroversial. The high-level strategic decision to bet on Rust has already been made, and the cost of depending on the language is already sunk. Now that we're past that point, I haven't heard anyone arguing why we shouldn't opt for memory safety when writing new standalone code. If there are people out there who disagree and think they have the arguments/clout/allies to make the case, please speak up. From my anecdotal experiences, I've heard two similar refrains: 1) "I haven't learned Rust well enough to feel confident choosing it for this code." 2) "I don't want to spend time learning Rust that I could be spending just writing the code in C++." I believe that every developer that writes C++ at Mozilla should be given access to enough Rust training and work hours to spend learning it beyond the training so that we can eliminate case #1. With the Rust training sessions at prior All-Hands and self-motivated learning, I think we've pretty well saturated the group of early adopters. These people are actively writing new Rust code. We need to at least get the people that want to learn Rust but don't feel like they've had time to that same place. I've been at (maybe half) a rust training at an allhands, and recently found myself looking at writing some code in rust. The experience was more about understanding other people's code, and re-using parts of it. Given that experience, I'd like to ask for a few more things: Readable Rust. We spent half-n-hour on 3 lines of code, and that shouldn't be like that. I'm not sure if that was because the code was written badly, or because reading rust code requires dedicated training. Copy-n-paste Rust code. That, to my experience, doesn't work like in any other language we frequently use. I'm used to copy, cut out the thing that it did originally, and the incrementally fill in my stuff. That works in most languages, but in Rust, it seems to break really really bad. Documentation improvements. I've hit quite a few documentation pieces that stated the existence of the thing I was looking for. My place of failure was rc, which seems to have gotten quite a bunch of doc updates in the meantime, which is good. I guess what I'm asking for is training on how to deal with rust code that other people wrote, maybe more so than writing rust code from scratch, starting with hello-world. Axel For case #2, there will always be people that don't want to learn new languages, and I'm sympathetic to their perspective. Learning Rust well does take a large investment of time. I don't know that I would go down the road of making Rust training mandatory (yet), but we are quickly going to hit a point where "I don't feel like learning Rust" is not going to cut it anymore. I would hope that by that point we will have trained everyone well enough that case #2 no longer exists, but if not we will have to make harder choices. The tradeoffs come when the code is less standalone, and we need to weigh the integration costs. This gets into questions like whether/how Rust code should integrate into the cycle collector or into JS object reflection, which is very much a technical decision that should be made by experts. I have a decent sense of who some of those experts might be, and would like to find the most lightweight mechanism for them to steer this ship. We definitely need to figure out an ergonomic solution for writing core DOM components in Rust, but I agree that this needs a fair bit of work to be feasible. Most of the situations I've seen recently were not that tightly integrated into Gecko. -Ted ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Scope of XML parser rewrite?
Am 24.05.17 um 09:34 schrieb Anne van Kesteren: On Tue, May 23, 2017 at 8:23 PM, Eric Rahm <er...@mozilla.com> wrote: I was hoping to write a more thorough blog post about this proposal (I have some notes in a gist), but for now I've added comments inline. The main takeaway here is that I want to do a bare-bones replacement of just the parts of expat we currently use. It needs to support DTD entities, have a streaming interface, and support XML 1 v4. That's it, no new features, no rewrite of our entire XML stack. "XML5" supports entities (at least my original version did), I think the main problem is that there's no support for external DTDs. Not sure how much that differs from parsing the internal subset. Either way, that's always been a feature that as far as the web is concerned is not supported so could conceivably be a Firefox-only thing. Only XUL needs it. Technical correction, our use of DTDs is independent of XUL, we use the same thing for XHTML UI parts. Which is the reason why we're not using HTML there. We do intend to get rid of it, that's what L20n and Fluent are for, and we're more than happy to see more people fight for that :-) Truth be told, though, we can only drop support when the last bit of UI is converted to L20n, and not just in Firefox, but also the other stuff. Y'know, Thunderbird, too, I guess. Axel My current goal is a drop-in replacement for expat with just the features gecko cares about, so just 1.0 version 4 I guess. It's possible whatever we end up with could be merged with another library when XML5 is settled, but I don't want to wait for that. Contrary to Henri, I think XML 1.0 edition 5 (which isn't "XML5") is worth considering given https://bugzilla.mozilla.org/show_bug.cgi?id=501837. It's what Chrome ships and our current implementation doesn't seem to align with either the 4th or 5th edition of XML 1.0. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Scope of XML parser rewrite?
Am 23.05.17 um 16:01 schrieb Daniel Fath: So, if I understand this correctly - We'll first need to land this component in Firefox, right? And if it proves itself fine, then formalize it. I was thinking of having resolutions for the issues that are currently warnings in red and multi-vendor buy-in. (Previously, Tab from Google was interested in making SVG parsing non-Draconian, but I have no idea how reflective of wider buy-in that remark was.) You also mentioned warnings in red and multi-vendor buy-in. What does that entail? Will lack of support for DTD be a problem? In XML5 it was decided, that instead of parsing DTD we just store list of named character references from https://html.spec.whatwg.org/multipage/syntax.html#named-character-references. While we could add another list and expand entities.json. It's possible I need to update spec to reflect that. Yes, not parsing DTDs would be a deal-breaker for the foreseeable future, as we're abusing DTDs to localize X(H)TML documents. Axel PS. I hope I'm not spamming you guys too hard, I'm kind of new to the mailing list thing. Daniel Fath, daniel.fa...@gmail.com ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Future of out-of-tree spell checkers?
Am 22.03.17 um 15:39 schrieb Jorge Villalobos: On 3/22/17 8:10 AM, Henri Sivonen wrote: On Wed, Mar 22, 2017 at 3:52 PM, Nicolas B. Pierron <nicolas.b.pier...@mozilla.com> wrote: On 03/22/2017 09:18 AM, Henri Sivonen wrote: Without XPCOM extensions, what's the story for out-of-tree spell checkers? […], which implements mozISpellCheckingEngine in JS and connects to the libvoikko[1] back end via jsctypes. […] Would compiling libvoikko to WebAssembly remove the need for jsctypes and XPCOM? It would remove the need for jsctypes, but how would a WebAssembly program in a Web Extension get to act as a spell checking engine once extensions can no longer implement XPCOM interfaces (mozISpellCheckingEngine in this case)? Note there is a bug on file to implement an spell-checker API for WebExtensions: https://bugzilla.mozilla.org/show_bug.cgi?id=1343551 The API request was approved but is low priority. Jorge Note, that bug seems about using an API like mozISpellCheckingEngine from web extensions. It doesn't seem to be about providing an implementation of it via a web extension. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
fyi, ./mach hangs on terminal-notifier? brew update to 1.7.1
Hi, as an fyi, I almost filed a bug on mach hanging on terminal-notifier after the end of a build or packaging step. Seems that was a bug in terminal-notifier 1.7.0, another brew update/upgrade updated that to 1.7.1 and fixed it. Just in case you've been in that boat. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Removal of B2G from mozilla-central
On 04/10/16 17:40, Fabrice Desre wrote: On 10/04/2016 08:34 AM, Axel Hecht wrote: I'd favor to remove at least anything related to l10n from b2g. It never really worked, and is a half-maintained copy of the almost-working stuff in mobile. In my local branches that try to create a test on broken l10n infrastructure, both mobile and b2g show up, and my preferred way to fix b2g would be to remove it (the l10n parts). What b2g specific l10n parts are you talking about? Can you provide links? https://dxr.mozilla.org/mozilla-central/source/b2g/locales/jar.mn has a bunch of overrides, and many of those are wrong also for fennec. b2g isn't as horrible as mobile is as it doesn't use the "browser" chrome package, but still. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Removal of B2G from mozilla-central
On 04/10/16 12:16, Gabriele Svelto wrote: * b2g ~20K lines which would also drop considerably due to the removal of the APIs, completely self-contained I'd favor to remove at least anything related to l10n from b2g. It never really worked, and is a half-maintained copy of the almost-working stuff in mobile. In my local branches that try to create a test on broken l10n infrastructure, both mobile and b2g show up, and my preferred way to fix b2g would be to remove it (the l10n parts). Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: MXR permanently offline, please transition to DXR
That will be really useful, thanks! All I need now is an open search engine for c-c to plug into Firefox search box. ;) Axel On 11 Jul 2016 03:09, "Peter Elmers" <pelm...@mozilla.com> wrote: > If it helps anyone, I happen to know that there exists an addon which > visits all anchor tags on a page and rewrites mxr.mozilla.org to > dxr.mozilla.org. > https://addons.mozilla.org/en-US/firefox/addon/mxr-to-dxr/ > > Disclaimer: I'm the author. > > On Wed, Jul 6, 2016 at 7:12 PM, Boris Zbarsky <bzbar...@mit.edu> wrote: > >> On 6/30/16 3:56 PM, Erik Rose wrote: >> >>> Hi, Kendall. As a pain mitigation strategy for MXR URLs embedded >>> immutably in Bugzilla and in people's Awesomebar histories, can we redirect >>> MXR requests as Dão suggests? >>> >> >> This really needs to happen. I've hit broken mxr links from bugzilla >> comments at least 7 or 8 times so far, and I was on vacation for most of >> the time mxr has been down. It's a pretty serious productivity drag. >> >> -Boris >> >> ___ >> dev-platform mailing list >> dev-platform@lists.mozilla.org >> https://lists.mozilla.org/listinfo/dev-platform >> > > > ___ > firefox-dev mailing list > firefox-...@mozilla.org > https://mail.mozilla.org/listinfo/firefox-dev > > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Questions about bindings for L20n
On 14/06/16 05:06, zbranie...@mozilla.com wrote: On Monday, June 13, 2016 at 9:39:32 AM UTC+1, Gijs Kruitbosch wrote: > Separately, the documentation put forth so far seems to indicate that the localization itself is also async, on top of the asyncness of the mutationobserver approach, and that could potentially result in flashes of unlocalized content, depending on "how" asynchronous that API really ends up being. (AFAIK, if the API returned an already-resolved promise, there might be less chance of that than if it actually went off and did IO off-main-thread, then came back with some results.) The DOM localization that is used in response to MutationObserver is sync. ... unless strings trigger a load, either if the initial suite of localizations isn't loaded yet, or the loaded strings trigger a runtime error, which requires more l10n files to be loaded. That's obviously cached, so it happens at first occasion. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Triage Plan for Firefox Components
Hi Emma, for those of us that are addicted to data: You have about a 1000 bugs of data, and I'd love to hear some of the good parts, and maybe also some of the bad parts. Also, you tested on three teams, and you report a success story from one. Could you frame that a bit? Is that within the expectations, or above or below? Axel On 29/03/16 22:07, Emma Humphries wrote: tl;dr In Quarter Two I'm implementing the work we’ve been doing to improve triage, make actionable decisions on new bugs, and prevent us from shipping regressions in Firefox. Today I’m asking for feedback on the plan which is posted at: https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko Allowing bugs to sit around without a decision on what we will do about them sends the wrong message to Mozillans about how we treat bugs, how we value their involvement, and reduces quality. The Firefox quality team (myself, Mike Hoye, Ryan VanderMeulen, Mark Cote, and Benjamin Smedberg) want to make better assertions about the quality of our releases by giving you tools to make clear decisions about which bugs must be fixed for each release (urgent) and actively tracking those bugs. What We Learned From The Pilot Program During the past 6 weeks, we have prototyped and tested a triage process with the DOM, Hello, and Developer Tools teams. Andrew Overholt, who participated in the pilot for the DOM team, said, “A consistent bug triage process can help us spread the load of watching incoming bugs and help avoid issues falling through the cracks." During the pilot, the DOM team uncovered critical bugs quickly so that people could be assigned to them. The pilot groups also found that the triage process needs to be fast and have tooling to make going through bugs fast. It’s easy to fall behind on triage for a component, but if you stay up to date it will take no more than 15 minutes a day. You can find the bugs we triaged during the pilot by looking for whiteboard tags containing ‘btpp-’. It is also important to have consistent, shared definitions for regression across components so triagers do not waste effort on mis-labeled bugs. Comments? I am posting this plan now for comment over the next week. I intend to finalize the triage plan for implementation by Tuesday, April 5th. Feedback and questions are welcome on the document, privately via email or IRC (where I’m emceeaich) or on the bugmast...@mozilla.org mailing list. Timeline January: finish finding component responsible parties February: pilot review of NEW bugs with four groups of components, draft new process Now: comment period for new process, finalize process Q2: implement new process across all components involved in shipping Firefox Q3: all newly triaged bugs following the new process -- Emma Humphries, Bugmaster ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to implement: W3C WebAppSec credentialmanagement API
Am Freitag, 11. März 2016 05:27:34 UTC+1 schrieb Martin Thomson: > On Fri, Mar 11, 2016 at 5:56 AM, Axel Nennker <ignisvul...@gmail.com> wrote: > > no password generation help by the UA > > I agree with MattN here, not doing this eliminates much of the > advantage of having a password manager. Or do you have a plan to rely > on sites doing that with CredentialContainer.store()? That doesn't > sound optimal to me. The questions is whether password generation should be part of the standard. The login manager / Firefox can still add password generation. I am not per se against adding password generation to the standard but there should be a PR that WebAppSec agrees on quickly. Support by major / popular sites would help. I fear that a password-complexity-and-renewal-policy-description-language is something that grows out of hand easily. The spec should focus on simple-and-useful features. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Intent to implement: W3C WebAppSec credentialmanagement API
Summary: This API is enabling a website to request a user's credentials from a user agent, and helps the user agent to correctly store user credentials for future use. It helps the LoginManager to get rid of most of the heuristics. Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1156047 Link to standard: https://w3c.github.io/webappsec-credential-management/ Platform coverage: Desktop first, every platform that today has LoginManager Current code is here: https://github.com/AxelNennker/firefox_credentials/ Estimated or target release: spec is still work in progress Preference behind which this will be implemented: not implemented yet DevTools bug: Suggested Additions: none, minimal viable API e.g. no password generation help by the UA. Federated Identity support is probably removed in the next version. Intend to ship in Chrome: https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/7ouLjWzcjb0 Web designer / developer use-cases: examples are in the spec ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposing preferring Clang over GCC for developer buidls
On 03/03/16 01:57, Jeff Gilbert wrote: On Wed, Mar 2, 2016 at 3:45 PM, Mike Hommey <m...@glandium.org> wrote: More importantly, changing the official toolchain has implications on performance. Sorry, I meant for general automation. Our final spins (especially LTO/PGO builds) should remain whatever gives us maximum perf. (not making any claims myself here!) Our PGO/LTO builds can take 10x+ what our normal integration builds take if it nets us a few percentage points of runtime perf. I suppose it becomes a question of divergence between fast-building builds and 'final' PGO/fully-optimized builds. We already have this to some extent with PGO vs non-PGO builds. This is gonna conflict with the release-promotion work that releng is doing now. We stop doing dedicated builds for at least beta now, and instead just take a known good CI build, and ship it. I also think that we should keep our CI builds close to what we intend to ship, for compiler/compiler-dependent bugs. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: rr chaos mode update
On 16/02/16 03:15, Kyle Huey wrote: Seems like a good thing to expect developers to do locally today. Two concerns: What's the successs criteria here? Also, speaking as an occasional code contributor, newcomers and folks like me will probably give up on contributing patches earlier. Axel - Kyle On Mon, Feb 15, 2016 at 6:08 PM, Justin Dolske <dol...@mozilla.com> wrote: On 2/14/16 9:25 PM, Bobby Holley wrote: How far are we from being able to use cloud (rather than local) machine time to produce a trace of an intermittently-failing bug? Some one-click procedure to produce a trace from a failure on treeherder seems like it would lower the activation energy significantly. And with that... At some point, what about having all *new* tests be battle-tested by X runs of rr-chaos testing? If it passes, it's allowed to run in the usual CI automation. If it fails, it's not (and you have a handy recording to debug). Justin ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Bug Program Next Steps
se are bugs with crash, topcrash, regression, or dataloss keywords; crash logs linked in comments; references to mozregression results; and others. The bug should not remain in this state after your review of it. You’ll need to decide which of the following states you’ll move this bug into, if you can’t you’ll need to be taking action: such as getting someone to run mozregression, need info’ing a domain expert, looking at checkins, and whatever else techniques you have to get a bug reduced. Once you have an understanding of the bug, you should assign it to one of these states. I'd ask you to make additions to these states for two use-cases: Ran out of time As you're asking for this to be done on a daily basis, I'd expect it to be time-bound. We do have bugs that just take a day or two to figure out what the next step is. Scheme doesn't work well for this bug I think this is an important option as we're learning how this works out for us. I'm also generally concerned how UX bugs or crashes would fit into these buckets. UX bugs, and possibly any other flavor of ideation, have the majority of work associated with "should we do this or not". And crashes as a single crash stack are hardly ever actionable. Urgent Assigned to a developer, release tracking flags nominated, and set at priority `P1`. If the bug is being worked on by somebody from outside your core team, a team mentor should be assigned. All these need to be set for the bug when you assign it to this state. This state is for bugs you find in your daily review which need a developer immediately. If the bug is not in need of immediate attention, then your team’s process should land the bug in one of the following states. Backlog This is a NEW bug that the team acknowledges is a bug, but is not a current priority, but will consider taking on. If the bug contains regression, crash, topcrash, or similar keywords and metadata, then the team can explain why it’s not a high priority. Is a Bug, Not Prioritized This is a terminal state for a NEW bug. We acknowledge the bug exists, it affects people, but it is not important enough to warrant working on it. The team will review and accept patches from the community for this bug report. Closed This is a terminal state for a NEW bug. After review, the bug is not something that can be worked on, or describes existing and expected behavior. Other States There are other states we’re looking at for bugs. These will cover a bug as it’s worked on, as ASSIGNED as is doesn’t provide useful information as to progress; flag bugs that have stalled, or needing code reviews, or sheriff attention; bugs that are on a release train; and bugs with fixes in general or ESR version. Timeline Now: finish finding component responsible parties February: pilot review of NEW bugs with four groups of components, draft new process March: comment period for new process, finalize process Q2: implement new process across all components Q3: all newly triaged bugs following the new process Are there good metrics to use to evaluate the success and impact of this process? To pick up on what I said earlier, getting a category for "didn't work" could be one way to measure, or at least enable us to systematically investigate areas for improvements. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Just Autoland It
Piling on: I'm using mozreview mostly as an occasional patch author: Plus, I can schedule a try build. Minus, I need to bother the reviewer with a published request in order to do so. Resorted to add yet another hg extension to my local .hg/hgrc. My most frequent concern is that bugzilla and mozreview use jargon and UX flows that have nothing in common. I don't think that either are good or better in their own right, too. And the mapping of one to the other isn't documented. "I want to cancel a review, or r-" doc is non-existent to hard-to-find. I just randomly click buttons. Which is basically what I do whenever I want to do something. I have a clear idea and intention on what I want to show up on bugzilla, but not on what to do on reviewboard to get there. Which might just be a category of documentation that's not written yet. Why I consider that to be a problem is gonna be in a separate reply to a different post on this thread. Axel On 25/01/16 13:26, Honza Bambas wrote: Writing both as a patch author and a reviewer as well. - as a patch author I want a full control on when the patch actually lands (dependencies, any other timing reasons, that only the author knows the best), "Don't land yet" comment somewhere will easily be overlooked - as a reviewer I don't want to bare the decision to land or not to land, at all - I want to preserve the mechanism "r+ with comments", which means to let the patch be landed by the author after updated (means reviewer doesn't need to look over it again) - as an author I want to write comments about the patch (about the structure, what it does, why) to make the review simpler for the reviewer ; commit message description may not be the best place, right? - will it be possible to still be using hg patch queues? - I (and others too) am not fun of MozReview UI. As a reviewer I found it very hard to orient in it: - what is the difference between [Reviews] and [Diff] tab? what is exactly it's content - where exactly to click to start a reivew of a patch I want to review now? Is in the "Commits" table? And is it under "Diff" or "Reviews"? - how can I mark hunks/files are already reviewed (something I like on Splinter)? - how can I see only a single file diff and easily navigate between files? (as in Splinter) - few weeks ago I didn't even know how to give an r+!! it's hidden under the [Finish review...] *tab*? - simply said: the UI is everything but self-explanatory and highly unfriendly, until that is fixed I'm not much willing to use MozReview mainly as a reviewer -hb- On 1/22/2016 3:35, Gregory Szorc wrote: If you have level 3 source code access (can push to central, inbound, fx-team) and have pushed to MozReview via SSH, as of a few weeks ago you can now land commits from the "Automation" drop down menu on MozReview. (Before only the review request author could trigger autoland.) This means that anyone [with permissions] can land commits with a few mouse clicks! It will even rewrite commit messages with "r=" annotations with the review state in MozReview. So if someone does a drive-by review, you don't have to update the commit message to reflect that reviewer. Neato! I've gotten into the habit of just landing things if I r+ them and I think they are ready to land. This has startled a few people because it is a major role reversal of how we've done things for years. (Typically we require the patch submitter to do the landing.) But I think reviewer-initiated landing is a better approach: code review is a gate keeping function so code reviewers should control what goes through the gate (as opposed to patch authors [with push access] letting themselves through or sheriffs providing a support role for people without push access). If nothing else, having the reviewer land things saves time: the ready-to-land commit isn't exposed to bit rot and automation results are available sooner. One downside to autoland is that the rebase will happen remotely and your local commits may linger forever. But both Mercurial and Git are smart enough to delete the commits when they turn into no-ops on rebase. We also have bug 1237778 open for autoland to produce obsolscence markers so Mercurial will hide the original changesets when you pull down the rebased versions. There is also potential for some Mercurial or Git command magic to reconcile the state of MozReview with your local repo and delete local commits that have been landed. This is a bit annoying. But after having it happen to me a few times, I think this is a minor annoyance compared to the overhead of pulling, rebasing, rewriting commit messages, and pushing locally, possibly hours or days after review was granted. I encourage my fellow reviewers to join me and "just autoland it" when granting review on MozReview. gps ___ dev-platform m
Re: Improving blame in Mercurial
On 12/11/15 10:46 PM, Joshua Cranmer wrote: On 12/11/2015 5:17 PM, Gregory Szorc wrote: If you have ideas for making the blame/annotate functionality better, please capture them at https://www.mercurial-scm.org/wiki/BlamePlan or let me know by replying to this message. Your feedback will be used to drive what improvements Mercurial makes. A "reverse blame" feature that shows when a line in an old revision was deleted or changed in a newer revision is something I've desperately wanted. I just recently successfully used `hg grep --all` for that. Axel (Relatedly, I know a lot of you want a Mercurial repo with CVS history to facilitate archeology. I hope to have that formally established in Q1. Stay tuned.) Are you planning on letting comm-central attach to the CVS history as well? ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Using the Taskcluster index to find builds
On 12/1/15 3:48 PM, Chris AtLee wrote: Localized builds should be at e.g. gecko.v2.mozilla-central.latest.firefox-l10n.win32-opt And yes, once we've got the naming structure nailed down, wget-en-US should change to use the index. I would expect l10n nightlies to be under nightly? How does one distinguish nightlies from non-nightlies under mozilla-central.latest? Assuming that nightlies might end up there on occasion? Axel On Tue, Dec 1, 2015 at 5:22 AM, Axel Hecht <l...@mozilla.com> wrote: I haven't found localized builds and their assets by glancing at things. Are those to come? Also, I suspect we should rewrite wget-en-US? Or add an alternative that's index-bound? Axel On 11/30/15 9:43 PM, Chris AtLee wrote: The RelEng, Cloud Services and Taskcluster teams have been doing a lot of work behind the scenes over the past few months to migrate the backend storage for builds from the old "FTP" host to S3. While we've tried to make this as seamless as possible, the new system is not a 100% drop-in replacement for the old system, resulting in some confusion about where to find certain types of builds. At the same time, we've been working on publishing builds to the Taskcluster Index [1]. This service provides a way to find a build given various different attributes, such as its revision or date it was built. Our plan is to make the index be the primary mechanism for discovering build artifacts. As part of the ongoing buildbot to Taskcluster migration project, builds happening on Taskcluster will no longer upload to https://archive.mozilla.org (aka https://ftp.mozilla.org). Once we shut off platforms in buildbot, the index will be the only mechanism for discovering new builds. I posted to planet Mozilla last week [2] with some more examples and details. Please explore the index, and ask questions about how to find what you're looking for! Cheers, Chris [1] http://docs.taskcluster.net/services/index/ [2] http://atlee.ca/blog/posts/firefox-builds-on-the-taskcluster-index.html ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Using the Taskcluster index to find builds
I haven't found localized builds and their assets by glancing at things. Are those to come? Also, I suspect we should rewrite wget-en-US? Or add an alternative that's index-bound? Axel On 11/30/15 9:43 PM, Chris AtLee wrote: The RelEng, Cloud Services and Taskcluster teams have been doing a lot of work behind the scenes over the past few months to migrate the backend storage for builds from the old "FTP" host to S3. While we've tried to make this as seamless as possible, the new system is not a 100% drop-in replacement for the old system, resulting in some confusion about where to find certain types of builds. At the same time, we've been working on publishing builds to the Taskcluster Index [1]. This service provides a way to find a build given various different attributes, such as its revision or date it was built. Our plan is to make the index be the primary mechanism for discovering build artifacts. As part of the ongoing buildbot to Taskcluster migration project, builds happening on Taskcluster will no longer upload to https://archive.mozilla.org (aka https://ftp.mozilla.org). Once we shut off platforms in buildbot, the index will be the only mechanism for discovering new builds. I posted to planet Mozilla last week [2] with some more examples and details. Please explore the index, and ask questions about how to find what you're looking for! Cheers, Chris [1] http://docs.taskcluster.net/services/index/ [2] http://atlee.ca/blog/posts/firefox-builds-on-the-taskcluster-index.html ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Why do we flush layout in nsDocumentViewer::LoadComplete?
On 11/27/15 4:09 AM, Robert O'Callahan wrote: On Fri, Nov 27, 2015 at 3:59 PM, Boris Zbarsky <bzbar...@mit.edu> wrote: On 11/26/15 9:24 PM, Robert O'Callahan wrote: We've always done it, but I can't think of any good reasons. I've tried to fix this in the past and ran into two problems. The first problem was that some tests failed as a result. This is somewhat minor, really. The second problem, pointed out by the first, is that some tests stopped testing what they mean to be testing, because all of our reftests and crashtests assume layout gets flushed onload, so they can test dynamic behavior by doing stuff after that. See https://bugzilla.mozilla.org/show_bug.cgi?id=581685 for details. I haven't had a chance to get back and really figure this out, though we should. Mmmm. This could be a significant win! Rob I wonder, how much of the web could rely on this, given our tests do? Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Merge moved to Thursday 29th
I'm also commented in the bug: If we're doing uplifts, I'm not sure we're winning by uplifting pre-landed strings. Either way, I think the risk assessment of the patch should be that of the actual patch that uses the strings, not "just adding strings". Also, I'd appreciate an ETA for the patch, as that's also going in to the risk that a patch comes with. Axel On 10/29/15 1:41 PM, Sylvestre Ledru wrote: Please request the uplift. Under specific circumstances (like this one), we take string changes in aurora. Thanks Sylvestre Le 29/10/2015 13:39, Masatoshi Kimura a écrit : I missed two commits for 44 branch. https://treeherder.mozilla.org/#/jobs?repo=fx-team=6b3c99e54177 Uplift requests will not help because they are string changes. On 2015/10/29 5:10, Sylvestre Ledru wrote: Hello, Because we want to synchronize the release of 42 and 44 devedition (next Tuesday), we are planning to perform the merge tomorrow, Thursday. As a consequence, nightly = 45, aurora = 44 and beta = 43 (42 is already in release). This will give us enough time to validate the first aurora build. I apologize for the very late notice. Of course, we will be friendly with uplift requests. Sylvestre ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Default window size on a screen of 1920x1080 or 1280x1024?
Hi, we're trying to find out what the default window size would be for people on screens of 1920x1080 or 1280x1024. Sadly, I can't find the code that actually computes that for the heck of it, can anybody help? Background: We want to ensure that the new about:privatebrowsing has the panels stacked side-by-side, but that depends on the window size. So we're trying to find out common window sizes for people. Some conversation in https://bugzilla.mozilla.org/show_bug.cgi?id=1198287 Thanks Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Default window size on a screen of 1920x1080 or 1280x1024?
Thanks, exactly what I was looking for. Axel On 9/16/15 7:13 PM, Gavin Sharp wrote: See https://hg.mozilla.org/mozilla-central/annotate/3e8dde8f8c17/browser/base/content/browser.js#l1017 if you're wondering about Firefox specifically. Gavin On Wed, Sep 16, 2015 at 7:26 AM, Axel Hecht <l...@mozilla.com> wrote: Hi, we're trying to find out what the default window size would be for people on screens of 1920x1080 or 1280x1024. Sadly, I can't find the code that actually computes that for the heck of it, can anybody help? Background: We want to ensure that the new about:privatebrowsing has the panels stacked side-by-side, but that depends on the window size. So we're trying to find out common window sizes for people. Some conversation in https://bugzilla.mozilla.org/show_bug.cgi?id=1198287 Thanks Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Largest chunks of code that are likely to be removable?
On 6/30/15 9:13 AM, Mike Hommey wrote: On Mon, Jun 29, 2015 at 11:19:08PM -0700, Nicholas Nethercote wrote: Hi, I'm wondering what the largest chunks of code there are in the codebase that are candidates for removal, i.e. probably with a bit of work but not too much. One that comes to mind is rdf/ (see https://bugzilla.mozilla.org/show_bug.cgi?id=1176160#c5) though I don't have a good understanding of how much stuff depends on it, even having seen https://bugzilla.mozilla.org/show_bug.cgi?id=420506. See the dependencies of bug 833098. Mike Note, that bug has the dependencies to move rdf/ from mozilla-central into comm-central. mail has many more dependencies on RDF, I think. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Voting in BMO
I recall that at least one group actively uses votes to prioritize stuff. I can't really tell which one, I'm leaning towards devtools, but I don't have any data to back that up. I mostly remember because I was surprised. Also, for a component like devtools, I can see how it'd make sense. Axel On 6/10/15 12:09 AM, Mark Côté wrote: In a quest to simplify both the interface and the maintenance of bugzilla.mozilla.org, we're looking for features that are of questionable value to see if we can get rid of them. As I'm sure everyone knows, Bugzilla grew organically, without much of a road map, over a long time, and it experienced a lot of scope bloat, which has made it complex both on the inside and out. I'd like to cut that down at least a bit if I can. To that end, I'd like to consider the voting feature. While it is enabled on a quite a few products, anecdotally I have heard many times that it isn't actually useful, that is, votes aren't really being used to prioritize features fixes. If your team uses voting, I'd like to talk about your use case and see if, in general, it makes sense to continue to support this feature. Thanks, Mark ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Mozillabuild 2.0 ready for testing
Hi Ryan, good news. One thing that's a bit unfortunate from the l10n perspective is the svn drop. SVN is still used quite frequently to host the website localizations, so keeping that in would be helpful. Axel, who should probably give this a test run in reals on his VM. On 3/9/15 2:44 AM, Ryan VanderMeulen wrote: For the past many months, I have been working on some major updates to the Mozillabuild package in order to make it more developer-friendly and easily-maintainable in the future. I am proud to say that at this time, it is ready for more widespread testing. The latest test build can be downloaded from the link below: http://people.mozilla.org/~rvandermeulen/MozillaBuildSetup2.0.0pre4.exe sha1sum: 9edb5c51bdb5f9a5d86baf5046967d8940654353 Release notes are available below. A few general notes: * It is strongly advised that you *NOT* install this over a previous 1.x install. Changes have been made to the directory structure and underlying packages that could result in unexpected results should you choose to do so. * As is always the case when updating Mozillabuild, you should clobber after installing. * Bugs that you come across can be filed in the mozilla.org::MozillaBuild component. My goal is to ship the final release in two weeks, so any feedback you can provide now would be welcome! Thanks, Ryan - SIGNIFICANT CHANGES * Added support for MSVC2015 and dropped support for MSVC 2013, WinSDK 8.1, and MSVC Express Edition. - MSVC Community Edition is now the recommended free compiler option * Added minTTY 1.1.3 and enabled it as the default console. - Windows cmd.exe can be used by default by removing the 1 from |SET USE_MINTTY=1| near the top of start-shell.bat * Overhauls to the start-msvc* batch scripts that improve consistency and simplify maintenance. - To launch a plain terminal with no MSVC path setting, use start-shell.bat (was start-shell-l10n.bat in previous releases) * Updated Mercurial to version 3.3.2 and switched to the native python version. - Allows extensions like bzexport that rely on previously-unavailable python components to work correctly. - Enables faster future updates in the event of serious bugs or security issues. - Enabled extensions: blackbox, color, histedit, mq, pager, progress, purge, rebase, share, transplant - See the Known Issues section for regressions from this change. * Updated python to version 2.7.9. - Included packages: pip 6.0.8, setuptools 14.0, virtualenv 12.0.7 OTHER UPDATES/ADDITIONS/REMOVALS * Removed SVN * Updated 7-zip to version 9.20 * Updated bundled CA certs to those from NSS 3.17.4 * Updated emacs to version 24.4 * Updated MSYS to version 1.0.18 and various components to the latest available for MSYS1 * Updated wget to version 1.16.1 KNOWN ISSUES * Changes in behavior due to using minTTY instead of Windows cmd.exe * Problems with python scripts that can't find an hg executable in the path (bug 1128586, bug 1131048) ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Testing for expected crashes in both C++ and js, for printf checking
On Friday, February 20, 2015 at 12:57:44 PM UTC+1, Ted Mielczarek wrote: On Thu, Feb 19, 2015, at 06:32 AM, a...@mozilla.com wrote: On Tuesday, February 17, 2015 at 6:33:42 PM UTC+1, Ted Mielczarek wrote: On Tue, Feb 17, 2015, at 10:27 AM, Axel Hecht wrote: Hi, I'd like to write tests to validate my assumptions around what's an error and what's a warning for localized values going into nsTextFormatter::smprintf. Basically, the tests would start with a reference string, and then a more or less random modification of that string, and a check if the segments are in, or if it crashes [1]. So I'll need a .cpp core, and a wrapper that feeds it data and checks the output. Any suggestions on how to do that right? When you say crashes do you mean an actual program crash? If so, shouldn't we just be fixing these in nsTextFormatter? If you don't need to monitor actual crashes then a Gtest should work just fine here. -Ted I'm talking actual crashes, and I don't know how we would fix the text formatter. I'm glancing at http://mxr.mozilla.org/mozilla-central/source/xpcom/glue/nsTextFormatter.cpp#778, and I had no idea how to fix the C++ API there. bug 1133554#c2 is a bug where we got it wrong in en-US, too. Doesn't crash, but produces undesirable results. Note, folks using the stringbundle.formatStringFromName just use a wstring array. The only code paths affected are those that get the string, and explicitly call into nsTextFormatter from c++. Can you provide an example of a call that crashes? I still feel like the right outcome here is to make the code not crash. -Ted I just tweaked the test: diff --git a/xpcom/tests/TestTextFormatter.cpp b/xpcom/tests/TestTextFormatter.cpp --- a/xpcom/tests/TestTextFormatter.cpp +++ b/xpcom/tests/TestTextFormatter.cpp @@ -11,7 +11,7 @@ { int test_ok = true; - nsAutoString fmt(NS_LITERAL_STRING(%3$s %4$S %1$d %2$d %2$d %3$s)); + nsAutoString fmt(NS_LITERAL_STRING(%3$s %4$S %1$f %2$d %2$d %3$s)); char utf8[] = Hello; char16_t ucs2[]={'W', 'o', 'r', 'l', 'd', 0x4e00, 0xAc00, 0xFF45, 0x0103, 0x00}; int d=3; And then you get ../../dist/bin/TestTextFormatter Segmentation fault: 11 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Testing for expected crashes in both C++ and js, for printf checking
On Friday, February 20, 2015 at 6:34:07 PM UTC+1, Tom Tromey wrote: Axel == axel-4eJtQOnFJqFBDgjK7y7TUQ axel-4ejtqonfjqfbdgjk7y7...@public.gmane.org writes: Axel We can only do this in the compiler if we actually compiled each Axel localized version by itself. Yes, I see what you mean. Axel The other easy way to reduce impact here is to reduce the use of Axel nsTextFormatter, or create a replacement that doesn't crash. L20n Axel would be one, or maybe there's C++ template stuff that can taint Axel values with their original types. I don't think there is a compile-time solution to this part of problem, because the substitution is done at runtime. I think what is needed for this problem is a runtime check to verify that the translated format string is consistent with the primary one. Tom I do this to validate the thinking that I've applied to compare-locales, which does source-based checks of localizations against en-US. With that tool, I can strip bad strings during repackaging or language pack building. That doesn't solve all problems, but gets use 90% there. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Testing for expected crashes in both C++ and js, for printf checking
On Tuesday, February 17, 2015 at 6:33:42 PM UTC+1, Ted Mielczarek wrote: On Tue, Feb 17, 2015, at 10:27 AM, Axel Hecht wrote: Hi, I'd like to write tests to validate my assumptions around what's an error and what's a warning for localized values going into nsTextFormatter::smprintf. Basically, the tests would start with a reference string, and then a more or less random modification of that string, and a check if the segments are in, or if it crashes [1]. So I'll need a .cpp core, and a wrapper that feeds it data and checks the output. Any suggestions on how to do that right? When you say crashes do you mean an actual program crash? If so, shouldn't we just be fixing these in nsTextFormatter? If you don't need to monitor actual crashes then a Gtest should work just fine here. -Ted I'm talking actual crashes, and I don't know how we would fix the text formatter. I'm glancing at http://mxr.mozilla.org/mozilla-central/source/xpcom/glue/nsTextFormatter.cpp#778, and I had no idea how to fix the C++ API there. bug 1133554#c2 is a bug where we got it wrong in en-US, too. Doesn't crash, but produces undesirable results. Note, folks using the stringbundle.formatStringFromName just use a wstring array. The only code paths affected are those that get the string, and explicitly call into nsTextFormatter from c++. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Testing for expected crashes in both C++ and js, for printf checking
On Thursday, February 19, 2015 at 11:27:11 PM UTC+1, Tom Tromey wrote: Axel writes: Axel I'm talking actual crashes, and I don't know how we would fix the Axel text formatter. I'm glancing at Axel http://mxr.mozilla.org/mozilla-central/source/xpcom/glue/nsTextFormatter.cpp#778, Axel and I had no idea how to fix the C++ API there. Axel bug 1133554#c2 is a bug where we got it wrong in en-US, too. Doesn't Axel crash, but produces undesirable results. Axel Note, folks using the stringbundle.formatStringFromName just use a Axel wstring array. The only code paths affected are those that get the Axel string, and explicitly call into nsTextFormatter from c++. I think the best answer for C++ is to write a compiler plugin to do format checking. GCC already has code to check that a format string matches the arguments. However, it can't be applied in several places in the tree because: * Various printf-likes in the tree extend the list of formats. For example, JS uses %hs to print a char16_t*. (And nsTextFormatter uses %S for this same thing, oops.) * The in-tree printf-likes sometimes use the same format characters as printf to mean different things. For example, nspr uses %l to mean 32-bit, whereas printf uses it to mean sizeof(long). * Some printf-likes, like nsTextFormatter, use a char16_t* as the format string. There are GCC bugs open about all of these. There's also the issue of looking up the format string in a message catalog. GCC has support for this, too, but I don't know offhand whether it can handle the approach used here. It's possible to fix all these things in the compilers. The advantage of a plugin is that nobody has to wait for a compiler upgrade to use it to find the bugs. I think it's worth doing. Before I learned about the various problems, I took a stab at using the GCC format attributes, and amidst all the nit-picking there are some genuine bugs. Tom We can only do this in the compiler if we actually compiled each localized version by itself. As long as we're using runtime files to switch languages, we can't rely on the compiler to check these things. There are two facets of l10n that we would loose if we went for a compiler-based solution: - language packs, i.e., addons that allow to install just another language. - repacks to create a localized build. This is mostly good use of resources, given that a repack takes about a minute or two per language. The other easy way to reduce impact here is to reduce the use of nsTextFormatter, or create a replacement that doesn't crash. L20n would be one, or maybe there's C++ template stuff that can taint values with their original types. Axel (PS: I wonder why I can't post to .platform via NNTP anymore :-/ ) ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Testing for expected crashes in both C++ and js, for printf checking
Hi, I'd like to write tests to validate my assumptions around what's an error and what's a warning for localized values going into nsTextFormatter::smprintf. Basically, the tests would start with a reference string, and then a more or less random modification of that string, and a check if the segments are in, or if it crashes [1]. So I'll need a .cpp core, and a wrapper that feeds it data and checks the output. Any suggestions on how to do that right? Axel [1] '%f' being the reference string, '%S' being the localization, pass in 5. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Deprecating localstore.rdf
On 8/4/14 9:55 PM, Benjamin Smedberg wrote: On 7/22/2014 8:47 AM, Roberto Agostino Vitillo wrote: Localstore.rdf will soon be replaced with a json store (see Bug 559505). I am currently planning to leave the localstore.rdf implementation as it is and issue a warning when a client tries to access to it. This is needed as some add-ons seem still to rely on it. We could use some Telemetry probes to see effectively how many add-ons are still using the rdf store once the patch lands. Are there any objections or remarks to the deprecation of localstore.rdf? This does involve a one-time import of localstore data into the new format, correct? I'm happy that we are doing this. I *believe* that this may be the last client of the RDF code in Firefox, which may allow us to remove RDF from Firefox in a future release. Do you already have an addon validation warning about addons using localstore? --BDS How much of XUL templates with RDF do we support still? Never kept track of templates to start with :-/ Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: .properties and a single \u
gandalf, stas, and I talked about this a bit, and we intend to settle with my mental parser plus a compare-locales warning: Broken escapes just pass through, \u - u, but we'll extend compare-locales to issue a warning on the l10n dashboard in those cases. The work is tracked in https://bugzilla.mozilla.org/show_bug.cgi?id=1040019. Axel On 7/11/14 4:26 PM, Axel Hecht wrote: Hi, in .properties foo = some \unicode bar = some \a creates the most icky output. I'd like to get a defined behavior, but it turns out to be hard. Java: - dies with a parsing error on foo, bar is some a XPCOM: - returns some , as \u is converted to \0 on foo, bar is some a Gaia: - returns some \unicode, some \a (doesn't drop unknown escapes apparently) Compare-locales: - dies with a python error on parsing in my code on foo, bar is some a My mental parser: - returns some unicode, some a I dislike both java and compare-locales, and I think that both gaia and xpcom don't work great. What's your take? Axel cross-posting to .platform for xpcom, .tools.l10n for gaia's l10n.js ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
.properties and a single \u
Hi, in .properties foo = some \unicode bar = some \a creates the most icky output. I'd like to get a defined behavior, but it turns out to be hard. Java: - dies with a parsing error on foo, bar is some a XPCOM: - returns some , as \u is converted to \0 on foo, bar is some a Gaia: - returns some \unicode, some \a (doesn't drop unknown escapes apparently) Compare-locales: - dies with a python error on parsing in my code on foo, bar is some a My mental parser: - returns some unicode, some a I dislike both java and compare-locales, and I think that both gaia and xpcom don't work great. What's your take? Axel cross-posting to .platform for xpcom, .tools.l10n for gaia's l10n.js ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Where is the document for jar.mn file format
On 7/2/14 12:25 PM, Yonggang Luo wrote: I am using Mozilla XUL SDK to build my own application, So I'd like to know what's the format of jar.mn file Took me a while to find it, but I think that https://ci.mozilla.org/job/mozilla-central-docs/Tree_Documentation/buildsystem/jar-manifests.html is the place to look. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Spring cleaning: Reducing Number Footprint of HG Repos
On 3/27/14, 12:53 AM, Taras Glek wrote: *User Repos* TLDR: I would like to make user repos read-only by April 30th. We should archive them by May 31st. Time spent operating user repositories could be spent reducing our end-to-end continuous integration cycles. These do not seem like mission-critical repos, seems like developers would be better off hosting these on bitbucket or github. Using a 3rd-party host has obvious benefits for collaboration self-service that our existing system will never meet. We are happy to help move specific hg repos to bitbucket. Once you have migrated your repository, please comment in https://bugzilla.mozilla.org/show_bug.cgi?id=988628so we can free some disk space. I think it's utterly sad making that we're giving up on hosting, instead of fixing it. I have several things in my user repos that only run on our hg server, mostly because all other repo hoster don't send correct mimetypes for raw files. In particular this affects dashboards I created to share aggregated bugzilla data etc. I'm also sad that we're removing the ability for contributors to share their mozilla-central clones, at least in large parts of the world. Pushing a full clone to some random server just isn't working for large parts of teh world. And all that while the opportunity for us to help you on the data consumption is just broken. Sad. Note, strategically, I think that mozilla needs to support developing o the web, and the github editor isn't it. It'll be web-based IDEs, which require good tooling and hosting to be on the same infrastructure. Axel *Non-User Repos* There are too many non-user repos. I'm not convinced we should host ash, oak, other project branches internally. I think we should focus on mission-critical repos only. There should be less than a dozen of those. I would like to stop hosting non-mission-critical repositories by end of Q2. This is a soft target. I don't have a concrete plan here. I'd like to start experimenting with moving project branches elsewhere and see where that takes us. *What my hg repo needs X/Y that 3rd-party services do not provide?* If you have a good reason to use a feature not supported by github/bitbucket, we should continue hosting your repo at Mozilla. *Why Not Move Everything to Github/Bitbucket/etc?* Mozilla prefers to keep repositories public by-default. This does not fit Github's business model which is built around private repos. Github's free service does not provide any availability guarantee. There is also a problem of github not supporting hg. I'm not completely sure why we can't move everything to bitbucket. Some of it is to do with anecdotal evidence of robustness problems. Some of it is lack of hooks (sans post-receive POSTs).Additionally, as with Github there is no availability guarantee. Hosting arbitrary Moz-related hg repositories does not make strategic sense. We should do the absolute minimum(eg http://bke.ro/?p=380) required to keep Firefox shipping smoothly and focus our efforts on making Firefox better. Taras ps. Footprint stats: *Largest User Repos Out Of ~130GB* 1.1Gdmt.alexandre_gmail.com 1.1Gjblandy_mozilla.com 1.1Gjparsons_mozilla.com 1.2Gbugzilla_standard8.plus.com 1.2Gmbrubeck_mozilla.com 1.2Gmrbkap_mozilla.com 1.3Gdcamp_campd.org 1.3Gjst_mozilla.com 1.4Gblassey_mozilla.com 1.4Ggszorc_mozilla.com 1.4Giacobcatalin_gmail.com 1.5Gcpearce_mozilla.com 1.5Ghurley_mozilla.com 1.6Gbsmedberg_mozilla.com 1.6Gdglastonbury_mozilla.com 1.6Gdtc-moz_scieneer.com 1.6Gjlund_mozilla.com 1.6Gsarentz_mozilla.com 1.6Gsbruno_mozilla.com 1.7Gmshal_mozilla.com 1.9Gmhammond_skippinet.com.au 2.1Glwagner_mozilla.com 2.4Garmenzg_mozilla.com 2.4Gdougt_mozilla.com 2.5Gbschouten_mozilla.com 2.7Ghwine_mozilla.com 2.8Geakhgari_mozilla.com 2.8Gmozilla_kewis.ch 2.9Grcampbell_mozilla.com 3.1Gbhearsum_mozilla.com 3.1Grjesup_wgate.com 3.2Gagal_mozilla.com 3.3Gaxel_mozilla.com 3.3Gprepr-ffxbld 4.2Gjford_mozilla.com 4.3Gmgervasini_mozilla.com 4.6Glsblakk_mozilla.com 5.0Gbsmith_mozilla.com 5.5Gnthomas_mozilla.com 5.8Gcoop_mozilla.com 6.5Gjhopkins_mozilla.com 7.7Graliiev_mozilla.com 9.2Gcatlee_mozilla.com 13Gstage-ffxbld *Space Usage by Non-user repos ~100GB* 24K integration/gaia-1_4 28K addon-sdk 28K projects/collusion 32K integration/gaia-1_1_0 32K projects/emscripten 32K projects/Moz2D 32K releases/mozilla-b2g18_v1_1_0 144Kprojects/addon-sdk-jetperf-tests 268Kipccode 452Ktestpilot-l10n 500Kreleases/firefox-hotfixes 700Kprojects/python-nss 896Kschema-validation 1.2Mprojects/mccoy 1.4Mpyxpcom 2.4Mplatform-model 2.4M
Re: How to efficiently walk the DOM tree and its strings
Hi, translating DOM is a bit funky. Generally, you can probably translate block elements one by one, but you need to persist inline elements. You should mark up the inline elements in the string that you send to the translation engine, such that you can support inline markup changing the order. Something like You would think the a href=foofunkyness/a would strongrule/rule. could translate into strongRuling/strong would be the a href=foofunkyness/a, you would think. Are you intending to also localize tooltips and the like? Axel On 3/3/14, 8:28 PM, Felipe G wrote: Hi everyone, I'm working on a feature to offer webpage translation in Firefox. Translation involves, quite unsurprisingly, a lot of DOM and strings manipulation. Since DOM access only happens in the main thread, it brings the question of how to do it properly without causing jank. This is the use case that I'm dealing with in bug 971043: When the user decides to translate a webpage, we want to build a tree that is a cleaned-up version of the page's DOM tree (to remove nodes that do not contain any useful content for translation; more details in the bug for the curious). To do this we must visit all elements and text nodes once and decide which ones to keep and which ones to throw away. One idea suggested is to perform the task in chunks to let the event loop breathe in between. The problem is that the page can dynamically change and then a tree representation of the page may no longer exist. A possible solution to that is to only pause the page that is being translated (with, say, EnterModalState) until we can finish working on it, while letting other pages and the UI work normally. That sounds a reasonable option to me but I'd like to hear opinions. Another option exists if it's possible to make a fast copy of the whole DOM, and then work on this snapshot'ed copy which is not live. Better yet if we can send this copy with a non-copy move to a Worker thread. But it brings the question if the snapshot'ing itself won't cause jank, and if the extra memory usage for this is worth the trade-off. Even if we properly chunk the task, it is still bounded by the size of the strings on the page. To decide if a text node should be kept or thrown away we need to run a regexp on it, and there's no way to pause that midway through. And after we have our tree representation, it must be serialized and encodeURIComponent'ed to be sent to the translation service. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Changing how build automation interacts with the tree
Hi, I've watched you guys thinking for an hour ;-) Some comments from me. Yes to moving build flows that generate assets into the tree. Yes to having a way for developers to reproduce what automation does. Yes to having jobs being executed more on demand than on push, and having that have idempotent results. Sceptical on the vision that we'll see the end of inbounds. The interactions between test results and rebase don't seem to be trivial enough to me to hope for non-backout always-open trees via auto land. I'm having an 'oh noes' for single command called by automation. My main point here is the usefulness of logs generated. When you put all sequential and parallel tasks into one single wrapper process, you end up with one big log file on ftp, like today. And if anything happens, one needs to read that log and reverse engineer which characters in this log are stdout/stderr, and to which task they belong. I know I can't tell good from bad in our logs. OTH, you could have all the structure of the process being exposed in the automation and its reporting. If something goes wrong, you can tell the location of the problem in the process right away, you can drill down to the process task, and its dependencies. If I think of the problem, I'm thinking along these lines: Let's specify the process, as a DAG of serialized and parallized tasks, inside the tree, and have the automation run that as is (*). Offer developers a console-only hook to that fragment of the complete automation process, akin to integration tests. * while using buildbot, parallel tasks would need to be executed sequentially. I read the recent posts by Taras et al that buildbot isn't a solid requirement going forward. A few comments on mozharness. One of the earliest tasks it offered, IIRC, was multi-locale android builds. Sadly, it happens that it's not helping those developers that want to create and test multi-locale builds. It's monolithic deliverable isn't what developers need at the point when they test multi-locale builds, nor does it blend in to the developer's setup. Folks like rnewman were glad once I explained how to avoid using mozharness for their builds. To me that's a sign of an inadequate level of abstraction. And, as it's been mentioned all over the call, l10n repacks: Testing: repacks are hard to test, and they should be. They're designed to be infallible, so that, no matter what happens in a localization, they're producing runnable builds. A test is challenged to tell between a broken localization and a broken build system. We shouldn't overestimate the amount of errors in the build that end up in a build bustage, and which of those are actually test failures. And which are not generating build failures, but are bustages. One example would be broken locale merge dirs, for example. Anything can be in those, and the builds build and run fine. They're just not showing the right strings. More generally, repacks are basically unowned at this point. There's a bit of ownership in build, in releng, and me, as to how they're done. There's absolutely nothing as far as reporting goes. The agreement between John and me was if there's anything odd, file a bug on releng to dig in. That's as much as I can get out of my brain into writing, I wish I had an hour-long video to go back and forth about stuff ;-) Axel On 2/28/14, 9:48 PM, Gregory Szorc wrote: (This is likely off-topic for many dev-platform readers. I was advised to post here because RelEng monitors dev-platform and I don't like cross-posting.) The technical interaction between build automation and mozilla-central has organically grown into something that's very difficult to maintain and improve. There is no formal API between automation and mozilla-central. As a result, we have automation calling into esoteric or unsupported commands and make targets. Change is difficult because it must be coordinated with automation changes. Build system maintainers lack understanding of what is and isn't used in automation. It's difficult to reproduce what automation does locally. The current approach slows everyone down, leads to too-frequent breakage (l10n repacks are a great example), and limits the efficiency of automation. I'm elated to state that at a meeting earlier today, we worked out a solution to these problems! Full details are in bug 978211. tl;dr we are going to marshal all interaction between automation and the tree through a mach-like in-tree script. This script will establish a clear, supported, and auditable API for automation tasks and will establish a level of indirection allowing the tree to change without requiring corresponding changes to automation. Some of the benefits of this approach include: * Abstracting the build backend away from automation. The tree will choose GNU make, pymake, mozmake, Tup, etc depending on what it knows is best. Currently, automation has {make, pymake, mozmake} hard-coded
Re: W3C Proposed Recommendations: RDF 1.1
As, still, module owner of RDF, I think that's the right thing for us to do. I haven't actually followed the development of the specs, but I'm positive that the development of those specifications doesn't impact us as a browser vendor. The impact of RDF is in the web application and addons system. Axel On 1/14/14 11:04 PM, L. David Baron wrote: There are eight W3C Proposed Recommendations for RDF 1.1 (two of which are actually Proposed Edited Recommendations): RDF Schema 1.1: W3C Proposed Edited Recommendation 09 January 2014 http://www.w3.org/TR/rdf-schema/ RDF 1.1 XML Syntax: W3C Proposed Edited Recommendation 09 January 2014 http://www.w3.org/TR/rdf-syntax-grammar/ RDF 1.1 N-Quads: W3C Proposed Recommendation 09 January 2014 http://www.w3.org/TR/n-quads/ RDF 1.1 N-Triples: W3C Proposed Recommendation 09 January 2014 http://www.w3.org/TR/n-triples/ RDF 1.1 Concepts and Abstract Syntax: W3C Proposed Recommendation 09 January 2014 http://www.w3.org/TR/rdf11-concepts/ RDF 1.1 Semantics: W3C Proposed Recommendation 09 January 2014 http://www.w3.org/TR/rdf11-mt/ RDF 1.1 TriG: W3C Proposed Recommendation 09 January 2014 http://www.w3.org/TR/trig/ RDF 1.1 Turtle: W3C Proposed Recommendation 09 January 2014 http://www.w3.org/TR/turtle/ There's a call for review to W3C member companies (of which Mozilla is one) open until February 9. If there are comments you think Mozilla should send as part of the review, or if you think Mozilla should voice support or opposition to the specification, please say so in this thread. (I'd note, however, that there have been many previous opportunities to make comments, so it's somewhat bad form to bring up fundamental issues for the first time at this stage.) My inclination is to explicitly abstain to indicate this is something we're not interested or involved in. -David ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: A proposal to reduce the number of styles in Mozilla code
Hi, two points of caution: In the little version control archaeology I do, I hit breaks blame for no good reason pretty often already. I'd not underestimate the cost for the project of doing changes just for the sake of changes. Tools don't get code right. I've seen various changes where tools reformat code they don't understand, and break it. Trailing whitespace is significant in some of our file formats, for example. Axel On 1/6/14 6:55 PM, Martin Thomson wrote: I think that this is a good start, but it doesn’t go quite far enough. Part of the problem with a policy that requires people to avoid reformatting of stuff they don’t touch is that it propagates formatting divergence. Sometimes because it’s better to conform to the style immediately adjacent the changed, but more so because it prevents the use of tools that reformat entire files (here, I’m not talking about things like names, but more whitespace conventions). For whitespace at least, I think that we need to do the following: 1. pick a style I really, really don’t care what this is. I’m thinking that we pick whatever people think that the current style is and give folks a fixed period to debate changes. 2. create some tools These tools should help people conform to the style. Primarily, what is needed is a tool with appropriate configuration that runs on the command line — e.g., mach reformat … clang-format is looking like a good candidate for C/C++, it just needs a configuration. For JavaScript, I’ve used the python js-beautify, but it’s pretty light on configuration options, it might need some hacking to make it better. Ideally though, there should be set of configuration files for common editors. I’m certain there are plenty out there already. Let’s bring them all together (attaching the files https://developer.mozilla.org/en-US/docs/Developer_Guide/Coding_Style might be enough). 3. reformat everything Take the command line tool and run it over all the code. I realise that this is the contentious part, but you don’t get the real benefits until you do this. Once you do this, it’s safe to reformat files at any time without messing with parts of files that you haven’t touched. This is important because many tools only reformat entire files. As for `hg blame` and related tools, I have a workaround for that. I’ve knocked together a tool that takes a file and a reformatter command line, and churns out a series of patches that retain blame: https://github.com/martinthomson/blame-bridge The patch bitrot problem is not easy to work around, but that will depend on how closely the affected files already conformed to the style guide. 4. enforce compliance This is probably a step for the future, but if there was - for example - a commit bot that waited for a clean build+test run, adding a format check to that run would allow the bot to block patches that screwed up the formatting of files. —Martin On 2014-01-05, at 18:34, Nicholas Nethercote n.netherc...@gmail.com wrote: We've had some recent discussions about code style. I have a propasal For the purpose of this proposal I will assume that there is consensus on the following ideas. - Having multiple code styles is bad. - Therefore, reducing the number of code styles in our code is a win (though there are some caveats relating to how we get to that state, which I discuss below). - The standard Mozilla style is good enough. (It's not perfect, and it should continue to evolve, but if you have any pet peeves please mention them in a different thread to this one.) With these ideas in mind, a goal is clear: convert non-Mozilla-style code to Mozilla-style code, within reason. There are two notions that block this goal. - Our rule of thumb is to follow existing style in a file. From the style guide: The following norms should be followed for new code, and for Tower of Babel code that needs cleanup. For existing code, use the prevailing style in a file or module, or ask the owner if you are on someone else's turf and it's not clear what style to use. This implies that large-scale changes to convert existing code to standard style are discouraged. (I'd be interested to hear if people think this implication is incorrect, though in my experience it is not.) I propose that we officially remove this implicit discouragement, and even encourage changes that convert non-Mozilla-style code to Mozilla-style (with some exceptions; see below). When modifying badly-styled code, following existing style is still probably best. However, large-scale style fixes have the following downsides. - They complicate |hg blame|, but plenty of existing refactorings (e.g. removing old types) have done likewise, and these are bearable if they aren't too common. Therefore, style conversions should do entire files in a single patch, where possible, and such patches should not make any non-style changes
Re: W3C Proposed Recommendations: Performance Timeline, User Timing, JSON-LD
I'll take a closer look at it. Will be interesting to see how this fits in with what we're planning, but so far, we didn't plan on taking it into account. - Original Message - From: Rob Campbell rcampb...@mozilla.com To: Till Schneidereit t...@tillschneidereit.net Cc: Boris Zbarsky bzbar...@mit.edu, Axel Kratel akra...@mozilla.com, dev-platform@lists.mozilla.org Sent: Wednesday, November 27, 2013 8:51:32 AM Subject: Re: W3C Proposed Recommendations: Performance Timeline, User Timing, JSON-LD I’ve been aware of these proposals. Honestly, there’s not a lot of meat in the performance timeline proposal and one of the editors doesn’t appear to be actively involved anymore. IMO, this is implementation detail for describing the data structures used by a timeline “entry”. I’m not sure how important a cross platform specification is, but would entertain the idea if there were good uses for it. ~ rob On Nov 26, 2013, at 05:56 , Till Schneidereit t...@tillschneidereit.net wrote: I don't know how well (or if at all) the current push for performance devtools is coordinated with these proposals, but it would certainly be worth looking into that. CCing Axel Kratel. On Tue, Nov 26, 2013 at 12:31 AM, Boris Zbarsky bzbar...@mit.edu wrote: On 11/25/13 7:07 PM, L. David Baron wrote: http://www.w3.org/TR/performance-timeline/ Performance Timeline http://www.w3.org/TR/user-timing/ User Timing Have we had anyone at all review these specs? My past experience with that working group and set of editors doesn't make me sanguine about them producing specs that can actually be implemented without reverse-engineering IE or Chrome If we _haven't_ had someone look at these before, we should do that now. And we probably need someone whose job description includes sanity-checking the stuff this working group produces. -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Add-on File Registration PRD
On 11/4/13 9:41 AM, Onno Ekker wrote: Jorge Villalobos wrote: Cross posting to dev.planning, where I originally intended this to be. Please follow up to dev.planning. Jorge On 10/30/13 3:42 PM, Jorge Villalobos wrote: Hello! As many of you know, the Add-ons Team, User Advocacy Team, Firefox Team and others have been collaborating for over a year in a project called Squeaky [1]. Our aim is to improve user experience for add-ons, particularly add-ons that we consider bad for various levels of bad. Part of our work consists on pushing forward improvements in Firefox that we think will significantly achieve our goals, which is why I'm submitting this spec for discussion: https://docs.google.com/document/d/1SZx7NlaMeFxA55-u8blvgCsPIl041xaJO5YLdu6HyOk/edit?usp=sharing The Add-on File Registration System is intended to create an add-on file repository that all add-on developers need to submit their files to. This repository won't publish any of the files, and inclusion won't require more than passing a series of automatic malware checks. We will store the files and generated hashes for them. On the client side, Firefox will compute the hashes of add-on files being installed and query the API for it. If the file is registered, it can be installed, otherwise it can't (there is planned transition period to ease adoption). There will also be periodic checks of installed add-ons to make sure they are registered. All AMO files would be registered automatically. This system will allow us to better keep track of add-on IDs, be able to easily find the files they correspond to, and have effective communication channels to their developers. It's not a silver bullet to solve add-on malware problems, but it raises the bar for malware developers. We believe this strikes the right balance between a completely closed system (where only AMO add-ons are allowed) and the completely open but risky system we currently have in place. Developers are still free to distribute add-ons as they please, while we get a much-needed set of tools to fight malware and keep it at bay. There are more details in the doc, so please give it a read and post your comments and questions on this thread. Jorge Villalobos Add-ons Developer Relations Lead [1] https://wiki.mozilla.org/AMO/Squeaky Hi, I have another use case which isn't clearly described by the current doc. I have an English version of Firefox/Thunderbird installed with additional language packs from http://ftp.mozilla.org/pub/mozilla.org/%APP%/%CHANNEL%/%VERSION%/%OS%/xpi/. After each update I have to manually add the language packs again. Those files are created by Mozilla but aren't published to amo. It would be a real shame if it wouldn't be possible anymore to add different languages to your installation. Onno Most language packs are featured on AMO these days, please check https://addons.mozilla.org/En-us/firefox/language-tools/. They're pulled from ftp by a admin tool in amo before the release. But yes, I'll actually need to read the original post with l10n in mind. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Add-on File Registration PRD
Hi, read the thread now. I ignored it based on the subject, btw, didn't seem to affect anything in real life from just glancing at that. I'd like to get langpacks excluded. Maybe we need to make their abilities to do stuff more robustly checked, but for a localizer wanting to test their work, this is really cumbersome. Note, the default config of those might even fail the malware checks, as the default identifies the author as mozilla.org. For non-l10n questions: We'd need developers to grant us a license to their code, right? Do we know for how many add-ons we'd actually need a license agreement that's not covered by the EULA of the add-on? I think that the current proposals for developers and internal org add-ons are too course. The proposal seems to expose them to malware just for the sake of their own development or deployment. I think that blocking the install is too hard for non-registered add-ons. A consistent UI that encourages users to uninstall non-registered add-ons might be all we need to get developers to register voluntarily. Also, the just break the network path seems to be easy to get to for malware installed by .exe installers on windows, at least. Or at least be open to social engineering as much as dismissing a non-registered add-on UI. Axel On 10/30/13 10:55 PM, Jorge Villalobos wrote: Cross posting to dev.planning, where I originally intended this to be. Please follow up to dev.planning. Jorge On 10/30/13 3:42 PM, Jorge Villalobos wrote: Hello! As many of you know, the Add-ons Team, User Advocacy Team, Firefox Team and others have been collaborating for over a year in a project called Squeaky [1]. Our aim is to improve user experience for add-ons, particularly add-ons that we consider bad for various levels of bad. Part of our work consists on pushing forward improvements in Firefox that we think will significantly achieve our goals, which is why I'm submitting this spec for discussion: https://docs.google.com/document/d/1SZx7NlaMeFxA55-u8blvgCsPIl041xaJO5YLdu6HyOk/edit?usp=sharing The Add-on File Registration System is intended to create an add-on file repository that all add-on developers need to submit their files to. This repository won't publish any of the files, and inclusion won't require more than passing a series of automatic malware checks. We will store the files and generated hashes for them. On the client side, Firefox will compute the hashes of add-on files being installed and query the API for it. If the file is registered, it can be installed, otherwise it can't (there is planned transition period to ease adoption). There will also be periodic checks of installed add-ons to make sure they are registered. All AMO files would be registered automatically. This system will allow us to better keep track of add-on IDs, be able to easily find the files they correspond to, and have effective communication channels to their developers. It's not a silver bullet to solve add-on malware problems, but it raises the bar for malware developers. We believe this strikes the right balance between a completely closed system (where only AMO add-ons are allowed) and the completely open but risky system we currently have in place. Developers are still free to distribute add-ons as they please, while we get a much-needed set of tools to fight malware and keep it at bay. There are more details in the doc, so please give it a read and post your comments and questions on this thread. Jorge Villalobos Add-ons Developer Relations Lead [1] https://wiki.mozilla.org/AMO/Squeaky ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Cost of ICU data
On 10/17/13 12:02 PM, Gervase Markham wrote: On 16/10/13 16:02, Axel Hecht wrote: We'll need to go down a path that works for Firefox OS. With Firefox OS, we don't have the download-size issue, do we? So we can ship all the data. Gerv We have issues with disk space, currently. We're already in the situation where all our keyboard data doesn't fit on quite a few of the devices out there. Also, FOTA size matters a bit, though that's probably less of a problem. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Cost of ICU data
On 10/16/13 5:39 PM, Jeff Walden wrote: On 10/16/2013 02:10 PM, Axel Hecht wrote: I wonder how far we can get by doing something along the lines we use for webfonts, starting to do the best we can with the data we already have, and improve once the perfect data is local. Having the Intl.Foo algorithms returning different data over time is, IMO, even worse than deciding that certain locales are less important than others. Aside from Math.random, of course, I can't think of anything in JS that has different behavior on the same inputs over time. Jeff For one, I don't think that's true for web. You might think so in terms of stuff in the js specs, but the distinction between that and html5 and all kinds of server errors and timing differences is just theory. More importantly, the impact of supporting a finite set of languages can easily be the nail in the coffin for the others. I don't think that's what mozilla stands for. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Cost of ICU data
On 10/17/13 2:41 PM, Dao wrote: On 16.10.2013 17:02, Axel Hecht wrote: We'll need to go down a path that works for Firefox OS. [...] But, yes, I think we'll need a hosted service to provide that data on demand in the end. This sounds like a non-starter for mobile devices, doesn't it? Well, it makes the implementation trickier. Of course, telefonica just updated the phones from 1.0.1 to 1.1 in spain, over the air without charges, so the infrastructure is there. It's an organizational effort to tie into that infrastructure. We'll need a reference implementation like we have with software update, and then get the our partner contacts in shape to explain how to do that on their side. Plus customizable hooks, of course. And then, yes, we'd need to still disable the downloads, or make them really optional, if you're on roaming data or something. But software update can do that already, too, I suspect. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Cost of ICU data
On 10/17/13 3:41 PM, Brian Smith wrote: On Thu, Oct 17, 2013 at 3:46 AM, Axel Hecht l...@mozilla.com wrote: We have issues with disk space, currently. We're already in the situation where all our keyboard data doesn't fit on quite a few of the devices out there. Where can one read more about this? This ICU data is not *that* huge. If we can't afford a couple of megabytes now on B2G then it seems like we're in for severe problems soon. Isn't Gecko alone growing by megabytes per year? I wish there were docs and clear cuts. We've been in dire problems already, when our QA smoketest phones wouldn't get updates for days due to system.img being too large. And thus we didn't get QA to run tests. These are the questions I asked last time, and don't have answers to: - What exactly are the limiting sizes? -- image size (per bootloader?) -- disk partition size --- at which point in time? user dependent? --- can we have telemetry for this, if so? I suspect we're talking about the joint size for gaia and gecko, but I'm not sure that's the case, or at least always the case. I.e., do we get a cookie if we move data from gaia into gecko? There's probably more that I don't know, just because I don't know much about phones, and the various processes to get software on to them. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Cost of ICU data
Jumping in late, so top posting. I think being able to load language data dynamically is a good idea. I don't see a reason why this should be tied in to a language pack, though. The other way around is a different question. i.e. language data doesn't include UI localization UI localization should include language data We have several multi-language products by now, those should work, in particular Firefox OS. We're doing quite a few things there that already duplicate language data. Much of that is in /shared, which isn't shared, but copied to many apps. Having that data inside gecko would actually get it to be shared. I think much of the ICU data (which is technically CLDR data packed in ICU mostly) flows along similar lines of our hyphenation dictionaries. The web should just work, independent of which UI locale you're using. I wonder how far we can get by doing something along the lines we use for webfonts, starting to do the best we can with the data we already have, and improve once the perfect data is local. I'm personally OK if this is a notification bar to reload, even. Axel PS: ICU is driven by js globalization api. That API was driven by MS and Google to get the data into their html app platforms. For mozilla, IMHO, the driver for g18n api should be Firefox OS, we're struggling to work around the lack of data for sorting, timezones, language data all around. On 10/15/13 6:06 PM, Benjamin Smedberg wrote: With the landing of bug 853301, we are now shipping ICU in desktop Firefox builds. This costs us about 10% in both download and on-disk footprint: see https://bugzilla.mozilla.org/show_bug.cgi?id=853301#c2. After a discussion with Waldo, I'm going to post some details here about how much this costs in terms of disk footprint, to discuss whether there are things we can remove from this footprint, and whether the footprint is actually worth the cost. This is particularly important because our user research team has identified Firefox download weight as an important factor affecting Firefox adoption and update rates in some markets. On-disk, ICU data breaks into the following categories: * collation tables - 3.3MB These are rules for sorting strings in multiple languages and situations. See http://userguide.icu-project.org/collation for basic background. These tables are necessary for implementing Intl.Collator. The Intl.Collator API has methods to expose a subset of languages. It is not clear from my reading of the specification whether it is expected that browsers will normally ship with the full set of languages or only the subset of the browser locale. * currency tables - 1.9 MB These are primarily the localized name of each currency in each language. This is used by the Intl.NumberFormat API to format international currencies. * timezone tables - 1.7MB Primarily the name of every time zone in each language. This data is necessary for implementing Intl.DateTimeFormat. * language data - 2.1 MB This is a bunch of other data associated with displaying information for a particular language: number formatting in various long and short formats, calendar formats and names for the various world calendar systems. == Do we need this data for any language other than the language Firefox ships in? Can we just include the relevant language data in each localized build of Firefox, and allow users to get other language data via downloadable language packs, similarly to how dictionaries are handled? Is it possible that some of this data (the collation tables?) should be in all Firefox locales, but other data (currency and timezone names) is not as important and we can ship it only in one language? As far as I can tell, the spec allows user agents to ship whatever languages they need; the real question is what users and site authors actually need and expect out of the API. (I'm reading the spec out of http://wiki.ecmascript.org/doku.php?id=globalization:specification_drafts) I am still working to get better number to quantify the costs in terms of lost adoption for additional download weight. Also, we are currently duplicating the data tables on mac universal builds, because they are compiled-in symbols. We should clearly use a separate file for these tables to avoid unnecessary download/install weight. This is now filed as bug 926980. --BDS ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Cost of ICU data
On 10/16/13 3:50 PM, Gervase Markham wrote: On 16/10/13 14:47, Anne van Kesteren wrote: The API is synchronous so that seems like a bad idea. As in, it'll cause the tab to freeze (one time only, when a new language is called for) while the file is downloading? OK, that's bad, but so is having Firefox be a lot bigger... Perhaps, as Brian suggested, we should be looking at using the Windows APIs and/or system ICU for some of this data, even if there are some things for which we want to ship our own implementation. Gerv We'll need to go down a path that works for Firefox OS. I think that being less-than-great at the first time you hit something off the main track is OK. We should see what actually happens with what's in the g18n apis now. We'll likely also need a way to free excessive use of disk space, or DOS attacks by sneaking up little fragments of language content for 200 languages or somesuch. But, yes, I think we'll need a hosted service to provide that data on demand in the end. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: What platform features can we kill?
On 10/11/13 2:47 PM, David Rajchenbach-Teller wrote: I'd be happy if we could progressively kill FileUtils.jsm and make nsIFile [noscript]. Don't know if this qualifies as platform feature, though. Cheers, David Both are heavily used in the js build system for gaia, fwiw. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: What platform features can we kill?
On 10/10/13 2:36 AM, Zack Weinberg wrote: On 2013-10-09 12:01 PM, Gervase Markham wrote: In the spirit of learning from this, what's next on the chopping block? In between keep the C++ implementation and scrap entirely is reimplement in JS, and I think that should be seriously considered for things like XSLT where there's no question but what it increases our attack surface, but there is also a strong (if small) constituency. Where it is currently impossible to do something in JS, that points at a weakness in the platform - whether capabilities or just speed. In that vein, I think we should take a hard look at the image decoders. Not only is that a significant chunk of attack surface, it is a place where it's hard to innovate; image format after image format has died on the vine because it wasn't *enough* of an improvement to justify the additional glob of compiled code. Web-deliverable JS image decoders could open that up. The other thing that comes to mind is, if Web Components lives up to its promise, perhaps it could be used for the built-in form controls? That's also a big pile of hair, and form elements in odd places have been an ongoing source of crasher bugs. zw I agree with the sentiment, but not on the eample. Having been a peer of the XSLT module back in the days, we started with a rather js DOM like implementation, and moved over to a pure nsIContent etc impl, and each step there won us an order of magnitude in perf. I don't think that XSLT is a good candidate for implementing it in JS. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: What platform features can we kill?
On 10/10/13 2:43 PM, Jeff Walden wrote: On 10/10/2013 02:27 PM, Axel Hecht wrote: I agree with the sentiment, but not on the eample. Having been a peer of the XSLT module back in the days, we started with a rather js DOM like implementation, and moved over to a pure nsIContent etc impl, and each step there won us an order of magnitude in perf. But do we actually care about the perf of sites that use XSLT now, as long as perf isn't completely abysmal? A utility company showing billing statements, I think we can slow down without feeling guilty. But if, say, Google Maps or whichever used XSLT (I seem to remember *something* Google used it, forcing Presto to implement XSLT, back in the day -- maybe they've switched now, blink thread might say if I checked it), we might care. Jeff My point is, the perf was completely abysmal, and the key is to use nsINodeInfo for the xpath patterns instead of DOM localName and namespaceURI string comparisons. There's also a benefit from using the low-level atom-nsID-based content creation APIs. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: What platform features can we kill?
On 10/9/13 6:18 PM, Boris Zbarsky wrote: On 10/9/13 12:01 PM, Gervase Markham wrote: In the spirit of learning from this, what's next on the chopping block? RDF Yes. I think that localstore.rdf is the long pole. Not so much because we abuse it for xul persistance, that's OK to fix. The thing that bothers me most is all of those addons that probably still use it. I'd love if we could get some data about that in particular, and RDF usage in addons in general. And then there's mailnews, of course. That one's sad. Close, but we moved everyone off of mailnews just before it got rid of RDF, IIRC. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: No more Makefile.in boilerplate
Hi, out of curiousity, I recall that relativesrcdir was actually the trigger to switch on and off some l10n functionality in jar packaging. Is that now on everywhere? Axel On 9/5/13 2:34 AM, Mike Hommey wrote: Hi, Assuming it sticks, bug 912293 made it unnecessary to start Makefile.in files with the usual boilerplate: DEPTH = @DEPTH@ topsrcdir = @top_srcdir@ srcdir = @srcdir@ VPATH = @srcdir@ relativesrcdir = @relativesrcdir@ include $(DEPTH)/config/autoconf.mk All of the above can now be skipped. Directories that do require a different value for e.g. VPATH or relativesrcdir can still place a value that will be taken instead of the default. It is not recommended to do that in new Makefile.in files, or to change existing files to do that, but the existing files that did require such different values still do use those different values. Also, if the last line of a Makefile.in is: include $(topsrcdir)/config/rules.mk That can be skipped as well. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Getting the current release version
On 8/30/13 4:14 PM, Ed Morley wrote: On 30 August 2013 15:09:08, Eric Shepherd wrote: This could even be a place in the source code we could pull up a MXR link and peel out of the code. I just don't know where in the code to get it. For platform: https://hg.mozilla.org/releases/mozilla-release/file/tip/config/milestone.txt For Firefox (and yeah currently the same as platform): https://hg.mozilla.org/releases/mozilla-release/file/tip/browser/config/version.txt Best wishes, Ed This is going to be off by one release for one week out of 6, though. We're doing the beta-release migration at release minus a week and a day, i.e., we're migrating on Sept 9, and release on Sept 17: https://mail.mozilla.com/home/ake...@mozilla.com/Release%20Management.html?view=monthdate=20130930 Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Getting the current UI locale from C++
On 8/29/13 12:03 PM, Henri Sivonen wrote: On Thu, Aug 29, 2013 at 10:12 AM, Henri Sivonen hsivo...@hsivonen.fi wrote: How do I get the language code for the currently active UI language pack from within Gecko C++ code in a way that works across desktop, Android, B2G and Metro? On IRC, I was pointed to https://mxr.mozilla.org/comm-central/source/mozilla/editor/composer/src/nsEditorSpellCheck.cpp#762 Does that mechanism work on B2G and Android? I'll read up on the other thread, and I still think the approach is wrong here, sorry. But yes, getting the selected locale for the global package is what I try to keep working at cost, notably on Android, where we're already doing stunts to do that. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Getting the current UI locale from C++
On 8/29/13 3:17 PM, Anne van Kesteren wrote: On Thu, Aug 29, 2013 at 1:48 PM, Axel Hecht l...@mozilla.com wrote: I'll read up on the other thread, and I still think the approach is wrong here, sorry. You'll have to explain that more fully I think. This is the current approach. However the current approach leads to all kinds of bugs because localization teams don't have expertise in this area. So we improve the status quo by reducing this source of bugs (that are still there and deployed to users). This is also the approach other browsers take. There might be a better approach (we'll have to research that, can't do it based on gut), but until that is there improving the status quo is a very good thing for our users. I followed up in https://bugzilla.mozilla.org/show_bug.cgi?id=910192 Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Feedback wanted on font checker
On 8/13/13 8:05 AM, Karl Tomlinson wrote: On Fri, 09 Aug 2013 13:51:45 +0200, Axel Hecht wrote: To clarify, the tool does support either .name. or .name-list. at this point. Is there a code path or a setup where we have for any language/family both a .name. and a .name-list. entry? I.e. pref(font.name.serif.zh-TW, Times); pref(font.name-list.serif.zh-TW, Droid Serif, Droid Sans Fallback); Just a randomly constructed example. The .name. and .name-list. are essentially concatenated into a single list, even though that is less clear in the second piece of code here: http://hg.mozilla.org/mozilla-central/annotate/c146d402a55f/gfx/thebes/gfxPlatform.cpp#l1019 http://hg.mozilla.org/mozilla-central/annotate/c146d402a55f/gfx/thebes/gfxFont.cpp#l4156 OK, that was easy to add, https://github.com/Pike/font-tool/commit/707bff7b6e87f695038bac1f80ba66e2b216593d. Thanks Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Feedback wanted on font checker
On 8/9/13 5:28 PM, Jonathan Kew wrote: On 9/8/13 15:36, Axel Hecht wrote: So I created three test cases based on the data I see, Greek and Bulgarian monospace and Hindi sans-serif. They're linked off of http://pike.github.io/fonts/. It's prerendered images on the left column, and regular text on the right. Hindi is blank squares as I expect, but for Bulgarian and Greek, I see Bulgarian and Greek letters on my 1.1 unagi. It'd be great to get some help as to what I actually see there. Oh fonts. Font fallback means that if there's ANY font on your Unagi that includes Cyrillic (or Greek) characters, you'll see them rather than blank boxes, even if that font isn't what's listed in the font.name prefs (And it may not even be monospaced... it's probably falling back to Open Sans or Charis SIL or something.) JK I've put a UnicodeHex font after the monospace or whatever now, so layout won't fallback beyond the monospace or sans-serif font families. Not sure if that's closer to what we actually want to test :-) Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Feedback wanted on font checker
On 8/9/13 1:27 PM, Karl Tomlinson wrote: Axel Hecht writes: On 8/8/13 10:45 PM, Karl Tomlinson wrote: Axel Hecht writes: On 8/8/13 5:17 PM, Jonathan Kew wrote: On 8/8/13 15:17, Axel Hecht wrote: Couter example seems to be Chinese, the unagi shows something, while my tool reports 13k missing glyphs for zh-TW. If we're using Droid Sans Fallback, I believe it supports somewhere well over 20,000 Chinese characters, so that probably includes all those in fontconfig's list - but that still doesn't mean it has *every* possible Chinese character. Yeah, I see DroidSansFallback in fonts.mk in moztt, https://github.com/mozilla-b2g/moztt/blob/master/fonts.mk#L31, but how would we pick that up? I suspect you want to add font.name-list support to your tool. These fonts should be searched by the product after font.name. pref(font.name-list.serif.zh-TW, Droid Serif, Droid Sans Fallback); pref(font.name-list.sans-serif.zh-TW, Roboto, Droid Sans, Droid Sans Fallback); pref(font.name-list.monospace.zh-TW, Droid Sans Fallback); Right. I pick up both, but not name-list as fallback to name. Basically, what comes last in all.js wins right now. Is there a reason why font.name.* isn't just the first in font.name-list.* ? I.e., why do we have two of them, and does it matter for what I'm trying to do? This was set up before I was around, so I'm not clear on the reasoning. Some comments [1] point out that only .name. is configurable in Firefox Preferences UI. Perhaps that would even support the idea of .name. remaining in the .name-list. in case some other font accidentally selected by the user was not helpful. I think you do want to check both preferences for the situation where a Latin font may have been inserted in .name. before a native font in .name-list. in an attempt to get better Latin characters. I suspect usually one font will cover the language, so your tool may not need to find the union the ranges or intersection of missing. At one stage IIRC only these fonts were being checked on Android, while desktop platforms had a fallback search through every font on the system. That has changed a bit with desktop platforms not always checking exhaustively, for perf reasons, and it may have changed on Android also. Even if other system fonts are checked in fallback, it is important that the .name. and .name-list. fonts cover their language, or you can get an awkward mix of characters from different fonts that don't work together. That can be as simple as a collage of different characters, or as bad as not working at all because some scripts require the presentation of characters to depend on neighbouring characters and this is not supported across a mix of fonts. [1] http://hg.mozilla.org/mozilla-central/annotate/e33c2011643e/layout/base/nsPresContext.cpp#l485 Yeah, we've seen those on our website a few times. Ughly, and a reason why I started writing this tool. To clarify, the tool does support either .name. or .name-list. at this point. Is there a code path or a setup where we have for any language/family both a .name. and a .name-list. entry? I.e. pref(font.name.serif.zh-TW, Times); pref(font.name-list.serif.zh-TW, Droid Serif, Droid Sans Fallback); Just a randomly constructed example. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Feedback wanted on font checker
On 8/9/13 1:51 PM, Axel Hecht wrote: On 8/9/13 1:27 PM, Karl Tomlinson wrote: Axel Hecht writes: On 8/8/13 10:45 PM, Karl Tomlinson wrote: Axel Hecht writes: On 8/8/13 5:17 PM, Jonathan Kew wrote: On 8/8/13 15:17, Axel Hecht wrote: Couter example seems to be Chinese, the unagi shows something, while my tool reports 13k missing glyphs for zh-TW. If we're using Droid Sans Fallback, I believe it supports somewhere well over 20,000 Chinese characters, so that probably includes all those in fontconfig's list - but that still doesn't mean it has *every* possible Chinese character. Yeah, I see DroidSansFallback in fonts.mk in moztt, https://github.com/mozilla-b2g/moztt/blob/master/fonts.mk#L31, but how would we pick that up? I suspect you want to add font.name-list support to your tool. These fonts should be searched by the product after font.name. pref(font.name-list.serif.zh-TW, Droid Serif, Droid Sans Fallback); pref(font.name-list.sans-serif.zh-TW, Roboto, Droid Sans, Droid Sans Fallback); pref(font.name-list.monospace.zh-TW, Droid Sans Fallback); Right. I pick up both, but not name-list as fallback to name. Basically, what comes last in all.js wins right now. Is there a reason why font.name.* isn't just the first in font.name-list.* ? I.e., why do we have two of them, and does it matter for what I'm trying to do? This was set up before I was around, so I'm not clear on the reasoning. Some comments [1] point out that only .name. is configurable in Firefox Preferences UI. Perhaps that would even support the idea of .name. remaining in the .name-list. in case some other font accidentally selected by the user was not helpful. I think you do want to check both preferences for the situation where a Latin font may have been inserted in .name. before a native font in .name-list. in an attempt to get better Latin characters. I suspect usually one font will cover the language, so your tool may not need to find the union the ranges or intersection of missing. At one stage IIRC only these fonts were being checked on Android, while desktop platforms had a fallback search through every font on the system. That has changed a bit with desktop platforms not always checking exhaustively, for perf reasons, and it may have changed on Android also. Even if other system fonts are checked in fallback, it is important that the .name. and .name-list. fonts cover their language, or you can get an awkward mix of characters from different fonts that don't work together. That can be as simple as a collage of different characters, or as bad as not working at all because some scripts require the presentation of characters to depend on neighbouring characters and this is not supported across a mix of fonts. [1] http://hg.mozilla.org/mozilla-central/annotate/e33c2011643e/layout/base/nsPresContext.cpp#l485 Yeah, we've seen those on our website a few times. Ughly, and a reason why I started writing this tool. To clarify, the tool does support either .name. or .name-list. at this point. Is there a code path or a setup where we have for any language/family both a .name. and a .name-list. entry? I.e. pref(font.name.serif.zh-TW, Times); pref(font.name-list.serif.zh-TW, Droid Serif, Droid Sans Fallback); Just a randomly constructed example. Axel So I created three test cases based on the data I see, Greek and Bulgarian monospace and Hindi sans-serif. They're linked off of http://pike.github.io/fonts/. It's prerendered images on the left column, and regular text on the right. Hindi is blank squares as I expect, but for Bulgarian and Greek, I see Bulgarian and Greek letters on my 1.1 unagi. It'd be great to get some help as to what I actually see there. Oh fonts. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Feedback wanted on font checker
Hi, I'm looking for a review of some rather hacky tool I just created to see if the fonts on b2g actually support a particular language. https://github.com/Pike/font-tool Basic outline of what the tool does: Parses langGroups.properties to see which locale has which group, with default to x-unicode. Preprocesses all.js with -DANDROID -DMOZ_WIDGET_GONK, and parses pref(font.name) Uses fc-scan to find all the fonts in moztt and platform/frameworks/base, pick those with style Regular. Uses fc-validate to figure out if those fonts actually support the given locale. Example, Bulgarian seems to be missing 60 glyphs for monospace: python buildfonts.py /src/central/mozilla-central/ bg monospace moztt/SourceCodePro-1.017/SourceCodePro-Regular.ttf:0 Missing 60 glyph(s) to satisfy the coverage for bg language sans-serif base/data/fonts/Roboto-Regular.ttf:0 Satisfy the coverage for bg language serif moztt/CharisSILCompact-4.114/CharisSILCompact-R.ttf:0 Satisfy the coverage for bg language Couter example seems to be Chinese, the unagi shows something, while my tool reports 13k missing glyphs for zh-TW. All of this has been mostly trial and error and stabbing in the dark, it'd be great if I could get some feedback and comments. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Feedback wanted on font checker
Hi Jonathan, thanks for the feedback, more inline. On 8/8/13 5:17 PM, Jonathan Kew wrote: On 8/8/13 15:17, Axel Hecht wrote: Hi, I'm looking for a review of some rather hacky tool I just created to see if the fonts on b2g actually support a particular language. https://github.com/Pike/font-tool Basic outline of what the tool does: Parses langGroups.properties to see which locale has which group, with default to x-unicode. Preprocesses all.js with -DANDROID -DMOZ_WIDGET_GONK, and parses pref(font.name) Uses fc-scan to find all the fonts in moztt and platform/frameworks/base, pick those with style Regular. Uses fc-validate to figure out if those fonts actually support the given locale. Example, Bulgarian seems to be missing 60 glyphs for monospace: python buildfonts.py /src/central/mozilla-central/ bg monospace moztt/SourceCodePro-1.017/SourceCodePro-Regular.ttf:0 Missing 60 glyph(s) to satisfy the coverage for bg language That sounds plausible, as SourceCodePro does not yet have Cyrillic support. (Do we have Droid Sans Mono on the device? If so, we should probably be falling back to that.) Nice, filed https://bugzilla.mozilla.org/show_bug.cgi?id=903038. sans-serif base/data/fonts/Roboto-Regular.ttf:0 Satisfy the coverage for bg language serif moztt/CharisSILCompact-4.114/CharisSILCompact-R.ttf:0 Satisfy the coverage for bg language Couter example seems to be Chinese, the unagi shows something, while my tool reports 13k missing glyphs for zh-TW. Not surprising, really. Two issues here: first, we rely on font fallback to find a font that supports a given character, if the default specified by prefs doesn't have it. In the case of Chinese, I think we tend to list a Latin font so that it will be used (rather than the often-ugly Latin glyphs in a Chinese font) for Latin characters, which are commonly found mixed in to Chinese pages, and then rely on fallback to find the actual Chinese font when needed. So you'd need to check the fallback font (probably Droid Sans Fallback, unless there's something else we're shipping on b2g), not necessarily the font listed in prefs. And second, validating the character coverage of a Chinese font is a bit tricky - you'll need to specify more carefully what the exact requirements are. For Chinese, there are tens of thousands of glyphs that are part of the writing system, but most everyday text will only use a relatively small subset - still several thousand, but nowhere near everything. The file fontconfig/tree/fc-lang/zh_tw.orth notes that it is Made by trimming the Big5 - unicode mapping down to just Chinese glyphs, which results in a list of around 13,000 characters. In contrast, according to [1] (though estimates will vary, no doubt), An educated Chinese person will know about 8,000 characters, but you will only need about 2-3,000 to be able to read a newspaper. So for most practical purposes, that collection should be (more than) sufficient. If we're using Droid Sans Fallback, I believe it supports somewhere well over 20,000 Chinese characters, so that probably includes all those in fontconfig's list - but that still doesn't mean it has *every* possible Chinese character. Yeah, I see DroidSansFallback in fonts.mk in moztt, https://github.com/mozilla-b2g/moztt/blob/master/fonts.mk#L31, but how would we pick that up? Axel JK [1] http://www.bbc.co.uk/languages/chinese/real_chinese/mini_guides/characters/characters_howmany.shtml ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Feedback wanted on font checker
On 8/8/13 10:45 PM, Karl Tomlinson wrote: Axel Hecht writes: On 8/8/13 5:17 PM, Jonathan Kew wrote: On 8/8/13 15:17, Axel Hecht wrote: Couter example seems to be Chinese, the unagi shows something, while my tool reports 13k missing glyphs for zh-TW. If we're using Droid Sans Fallback, I believe it supports somewhere well over 20,000 Chinese characters, so that probably includes all those in fontconfig's list - but that still doesn't mean it has *every* possible Chinese character. Yeah, I see DroidSansFallback in fonts.mk in moztt, https://github.com/mozilla-b2g/moztt/blob/master/fonts.mk#L31, but how would we pick that up? I suspect you want to add font.name-list support to your tool. These fonts should be searched by the product after font.name. pref(font.name-list.serif.zh-TW, Droid Serif, Droid Sans Fallback); pref(font.name-list.sans-serif.zh-TW, Roboto, Droid Sans, Droid Sans Fallback); pref(font.name-list.monospace.zh-TW, Droid Sans Fallback); Right. I pick up both, but not name-list as fallback to name. Basically, what comes last in all.js wins right now. Is there a reason why font.name.* isn't just the first in font.name-list.* ? I.e., why do we have two of them, and does it matter for what I'm trying to do? Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Three RDFa-related W3C Proposed (Edited) Recommendations
I've only quickly glanced at those, and I haven't followed those discussions at all, I have to admit. Are there any practical consequences for gecko/firefox? It doesn't look like it would, in particular when looking at the reference implementations being all on top of html platforms. Axel On 7/17/13 1:12 AM, L. David Baron wrote: The W3C has released three RDFA-related documents, one proposed recommendation: HTML+RDFa 1.1: http://www.w3.org/TR/2013/PR-html-rdfa-20130625/ and two proposed edited recommendations (which contain only editorial changes): RDFa 1.1 Core: http://www.w3.org/TR/2013/PER-rdfa-core-20130625/ XHTML+RDFa 1.1 http://www.w3.org/TR/2013/PER-xhtml-rdfa-20130625/ There's a call for review to W3C member companies (of which Mozilla is one) open until Tuesday, July 23 (one week from today). If there are comments you think Mozilla should send as part of the review, or if you think Mozilla should voice support or opposition to the specification, please say so in this thread. (I'd note, however, that there have been many previous opportunities to make comments, so it's somewhat bad form to bring up fundamental issues for the first time at this stage.) There was one formal objection earlier in the process, whose history is documented in http://lists.w3.org/Archives/Public/public-rdfa-wg/2013Jan/0057.html -David ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: review stop-energy (was 24hour review)
On 7/11/13 8:24 PM, Jeff Walden wrote: On 07/11/2013 03:09 AM, Nicholas Nethercote wrote: On Thu, Jul 11, 2013 at 7:48 AM, Jeff Walden jwalden+...@mit.edu wrote: Establishing one-day turnaround time on reviews, or on requests, would require a lot more polling on my review queue. You poll your review queue? Like, by visiting your Bugzilla dashboard, or something like that? That's *awful*. I personally use a push notification system called email with filters. Well, strictly speaking it's poll-like because I have to check my high priority bugs folder, but I do that anyway multiple times per day so I'm unlikely to take more than an hour or two (while working) to notice a review request. I have https://bugzilla.mozilla.org/request.cgi?requestee=jwalden%2Bbmo%40mit.edudo_union=1group=typeaction=queue open in a browser tab and check it from time to time. I don't see how that's any different from a mail-filtering folder except in terms of the UI. They're both polling, as you note. :-) Jeff I wish I could watch more than one requestee in one page. I actually have a tab with a history of three bugzilla accounts' requests page, and go back and forth and reload. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On 5/31/13 10:14 PM, Johnny Stenback wrote: On 5/31/2013 12:32 AM, Mike Hommey wrote: [...] Option 1 is where I personally think it's worth investing effort. It means we'd need to set up an atomic bidirectional bridge between hg and git (which I'm told is doable, and there are even commercial solutions for this out there that may solve this for us). Assuming we solve the bridge problem one way or another, it would give us all the benefits listed above, plus developer tool choice, and we could roll this out incrementally w/o the need to change all of our infrastructure at once. I.e. our roll out could look something like this: 1. create a read only, official mozilla-central git mirror 2. add support for pushing to try with git and see the results in tbpl 3. update tbpl to show git revisions in addition to hg revisions 4. move to project branches, then inbound, then m-c, release branches, etc Another way to look at this would be to make the git repository the real central source, and keep the mercurial branches as clones of it, with hg-git (and hg-git supports pushing to git, too). This would likely make it easier to support pushing to both, although we'd need to ensure nobody pushes octopus merges in the git repo. Yup, could be, and IMO the main point is that we'd have a lot of flexibility here. Option 2 is where this discussion started (in the Tuesday meeting a few weeks ago, https://wiki.mozilla.org/Platform/2013-05-07#Should_we_switch_from_hg_to_git.3F). Since then I've had a number of conversations and have been convinced that a wholesale change is the less attractive option. The cost of a wholesale change will be *huge* on the infrastructure end, to a point where we need to question whether the benefits are worth the cost. I have also spoken with other large engineering orgs about git performance limitations, one of which is doing the opposite switch, going from git to hg. I bet this is facebook. Their usecase includes millions of changesets with millions of files (iirc, according to posts i've seen on the git list). I've promised not to mention names here, so I won't confirm nor deny... but the folks I've been talking to mostly have a repo that's a good bit less than a single order of magnitude larger than m-c, so a couple of hundred k files, not millions. And given the file count trend in m-c (see attached image for an approximation), that doesn't make me feel too good about a wholesale switch given the work involved in doing so. I wouldn't be surprised if depth of tree was an impacting perf, given how git stores directories, with trees, tree children and refs. I would think that 1000 files in the top level dir perform better than a 1000 files in 20 dirs depth. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
Weirdly enough, I'm hoping we're using one or the other, and I think git is more promising. Yes, I need to rewrite a bunch of stuff l10n-wise, but still. I actually think that we should aim high. Don't bother about command lines, but what takes us to a system where people can just contribute to Firefox and Gecko on the web. Find the bug, click a button, start editing in your browser, try, review, merge. If we can get that working without involving a single thing but a browser, than we're making a change. git command line vs hg command line doesn't bring that change. The best model people have come up so far is a fork per bug (LegNeato at the time) or per user (github). I see how someone can serve a few 100k of forks (see our bugcount) on git storage, but I don't see that with hg. That to me is the compelling argument. Doing both hg and git sounds like we'll get the worst of both worlds. I'm also advocating for taking hosting dead serious. We're at least struggling with the amount of repos we're serving on hg right now, adding more complexity won't make the systems more stable. Also, Vicent (githubber) has a great talk about the mistakes they did based on git, http://vimeo.com/64716825 (30 mins). They're well past the approaches we're currently at on the hg side. If we don't have to be compatible with hg, we can also rethink the constraints that's putting on us for merge days, etc. I know that option 2 isn't a quick path, and I love the beauty of hg, but we've failed to use that beauty to make the next game changing infrastructure to support contributors, IMHO. I can see us having a better chance with git in the backend. Quick notes on hackability: In terms of stable and reliable hacking, hg and git are on par. Shell out to the command line tools, parse the output. hg being mostly in python is nice for python hacking, but the code paths you're hooking in to are far from stable. I do that extensively, and quite a few ports have been painful. Scripting hg outside of python, well, yes. git has libgit2 now, which is a very basic C impl, and jgit, a java implementation of git. Bindings for libgit2 exist in many languages, but only the ruby and C# are really good. In particular the python binding is far from being pythonic, and from being complete. If it's the right base to create a pythonic api is TBD. Regarding bugzilla integration, there are perl bindings that get modifications. I refuse to know perl good enough to make any statement on the value of the perl bindings, though. Axel On 5/31/13 2:56 AM, Johnny Stenback wrote: [TL;DR, I think we need to embrace git in addition to hg for Firefox/Gecko hacking, what do you think?] Hello everyone, The question of whether Mozilla engineering should embrace git usage for Firefox/Gecko development has come up a number of times already in various contexts, and I think it's time to have a serious discussion about this. To me, this question has already been answered. Git is already a reality at Mozilla: 1. Git is in use exclusively for some of our significant projects (B2G, Gaia, Rust, Servo, etc) 2. Lots of Gecko hackers use git for their work on mozilla-central, through various conversions from hg to git. What we're really talking about is whether we should embrace git for Firefox/Gecko development in mozilla-central. IMO, the benefits for embracing git are: * Simplified on-boarding, most of our newcomers come to us knowing git (thanks to Github etc), few know hg. * We already mirror hg to git (in more ways than one), and git is already a necessary part of most of our lives. Having one true git repository would simplify developers' lives. * Developers can use git branches. They just work, and they're a good alternative to patch queues. * Developers can benefit from the better merge algorithms used by git. * Easier collaboration through shared branches. * We could have full history in git, including all of hg and CVS history since 1998! * Git works well with Github, even though we're not switching to Github as the ultimate source of truth (more on that below). Some of the known issues with embracing git are: * Performance of git on windows is sub-optimal (we're already working on it). * Infrastructure changes needed... So in other words, I think there's significant value in embracing git and I think we should make it easier to hack on Gecko/Firefox with git. I see two ways to do that: 1: Embrace both git and hg as a first class DVCS. 2: Switch wholesale to git. Option 1 is where I personally think it's worth investing effort. It means we'd need to set up an atomic bidirectional bridge between hg and git (which I'm told is doable, and there are even commercial solutions for this out there that may solve this for us). Assuming we solve the bridge problem one way or another, it would give us all the benefits listed above, plus developer
Re: Replacing Gecko's URL parser
On 7/1/13 8:30 PM, Gavin Sharp wrote: .sOn Mon, Jul 1, 2013 at 10:58 AM, Benjamin Smedberg benja...@smedbergs.us wrote: Idempotent: Currently Gecko's parser and the URL Standard's parser are not idempotent. E.g. http://@/mozilla.org/ becomes http:///mozilla.org/ which when parsed becomes http://mozilla.org/ which is somewhat bad for security. My plan is to change the URL Standard to fail parsing empty host names. I'll have to research if there's other cases that are not idempotent. I don't actually know what this means. Are you saying that http://@/mozilla.org/; sometimes resolves to one URI and sometimes another? function makeURI(str) ioSvc.newURI(str, null, null) makeURI(http://@/mozilla.org/;).spec - http:///mozilla.org/ makeURI(http:///mozilla.org/;).spec - http://mozilla.org/ In other words, makeURI(makeURI(str).spec).spec does not always return str. Gavin nitpicking, that's not not idempotent. It's not round-tripping, but it looks like it's idempotent. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Replacing Gecko's URL parser
On 7/3/13 8:49 AM, Anne van Kesteren wrote: On Tue, Jul 2, 2013 at 12:09 PM, Benjamin Smedberg benja...@smedbergs.us wrote: Both resource: and chrome: have host names and need to support relative URIs. Neither of them is a candidate for standardization, though. We should just add them as special known schemes in our parser. Well, either we have to standardize their parsing behavior, limit their parsing behavior to chrome, or think of some third alternative. We do not want url = new URL(rel, base) to differ across engines for any rel or base. How many odd protocols and assumptions on how they work do we still have in mailnews' abuse of RDF? Not sure what's lurking in localstore.rdf and mimeTypes.rdf. Also, sorry, can't offer more help than asking these days. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Geolocation and the Mac
On 5/22/13 1:45 AM, Doug Turner wrote: In Bug 874587, we are considering using Core Location as the default geolocation provider on the Mac. This would replace the use of the NetworkGeolocationProvider (that currently points to GLS). After code reviews, we plan to enable this on Nightly and see how it goes. On Android, we already due use the system location provider and not the NetworkGeolocationProvider... so this isn't something unexpected. The main difference is that you will get one prompt from the OS the first time you use geolocation from Firefox -- just like every other standard Mac application. Does anyone have any concern that is specific to change from the NetworkGeolocationProvider to a Mac platform specific one? Asking the other way around, why are we doing this? Ad hoc it just looks like more code to maintain. Also, is there documentation on how the mac does geo location? We also make statements about our requirements on 3rd party location services in https://www.mozilla.org/en-US/legal/privacy/firefox.html. Depending on how mac locates, those may or may not hold? Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal for an inbound2 branch
On 4/30/13 8:46 AM, Gregory Szorc wrote: On 4/26/2013 12:17 PM, Ryan VanderMeulen wrote: Specific goals: -Offer an alternative branch for developers to push to during extended inbound closures -Avoid patch pile-up after inbound re-opens from a long closure Specific non-goals: -Reducing infrastructure load -Changing pushing strategies from the widely-accepted status quo (i.e. multi-headed approach) -Creating multiple integration branches that allow for simultaneous pushing (i.e. inbound-b2g, inbound-gfx, etc) My proposal: -Create an inbound2 branch identically configured to mozilla-inbound. -Under normal circumstances (i.e. m-i open), inbound2 will be CLOSED. -In the event of a long tree closure, the last green changeset from m-i will be merged to inbound2 and inbound2 will be opened for checkins. ---It will be a judgment call for sheriffs as to how long of a closure will suffice for opening inbound2. -When the bustage on m-i is resolved and it is again passing tests, inbound2 will be closed again. -When all pending jobs on inbound2 are completed, it will be merged to m-i. -Except under extraordinary circumstances, all merges to mozilla-central will continue to come from m-i ONLY. -If bustage lands on inbound2, then both trees will be closed until resolved. Tough. We apparently can't always have nice things. If you consider that every repository is essentially a clone of mozilla-central, what we have *now* is effectively equivalent to a single repository with multiple heads/branches/bookmarks. However, the different heads/branches/bookmarks differ in: * How much attention sheriffs give them. * The automation configuration (coalescing, priority, etc). * Policies around landing. * How developers use it. These are all knobs in our control. When we say create an inbound2, we're essentially establishing a new head/branch/bookmark that behaves much like inbound1 with a slightly different landing policy. If that's what we want to do, sure. I think it's better than a single, frequently closed inbound. Anyway, no matter how much I think about this proposal, I keep coming back to the question of why don't we use project branches more? Instead of everyone and her brother landing on inbound, what if more landings were performed on {fx-team, services-central, wood-named twig, etc}? I /think/ the worst that can happen is merge conflicts and bit rot. And, we can abate that through intelligent grouping of related commits in the same repository, frequent merges, and maybe even better communication (perhaps even automatically with tools that alert developers to potential conflicts - wouldn't it be cool if you updated a patch and Mercurial was like o hai - Ehsan recently pushed a Try push that conflicts with your change: you two should talk.). As a counter-proposal, I propose that we start shifting landings to project branches/twigs. We should aim for a small and well-defined set of repositories (say 3 to 5) sharing similar automation configuration and sheriff love. By keeping the number small, it's easy to figure out where something should land and it's not too much of an extra burden on sheriffs. We can still keep inbound, but it should only be reserved for major, cross-repository landings with multi-module impact (e.g. build system changes), merges from the main landing repositories (unless we merge straight to central), and possibly as a backup in case one of the landing repositories is closed. We can test this today with very little effort: we figure out how to bucket commits, re-purpose existing repositories/twigs, and see what happens. If it works: great - we've just validated that distributed version control works for Firefox development (as opposed to the CVS/Subversion workflow we're currently using with inbound). If not, we can try variations and/or the inbound2 idea. Is there sanity to this proposal or am I still crazy? To me the caveat here is that merge conflicts aren't restricted to VC merge conflicts. Changes to APIs in one branch with a new customer in another is probably the most frequent. Naming conflicts between two jsms in different modules, conflicting contract ids, etc. I think that the cost of getting to a single revision that works out of multiple independent branches in a piece of software as big and modular as firefox has to be significant. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Some data on mozilla-inbound
On 4/22/13 9:54 PM, Kartikaya Gupta wrote: TL;DR: * Inbound is closed 25% of the time * Turning off coalescing could increase resource usage by up to 60% (but probably less than this). * We spend 24% of our machine resources on changes that are later backed out, or changes that are doing the backout * The vast majority of changesets that are backed out from inbound are detectable on a try push Do we know how many of these have been pushed to try, and passed/compiled what they'd fail later? I expect some cost of regressions to come from merging/rebasing, and it'd be interesting to know how much of that you can see in the data window you looked at. has been pushed to try is obviously tricky to find out, in particular on rebases, and possibly modified patches during the rebase. Axel Because of the large effect from coalescing, any changes to the current process must not require running the full set of tests on every push. (In my proposal this is easily accomplished with trychooser syntax, but other proposals include rotating through T-runs on pushes, etc.). --- Long verion below --- Following up from the infra load meeting we had last week, I spent some time this weekend crunching various pieces of data on mozilla-inbound to get a sense of how much coalescing actually helps us, how much backouts hurt us, and generally to get some data on the impact of my previous proposal for using a multi-headed tree. I didn't get all the data that I wanted but as I probably won't get back to this for a bit, I thought I'd share what I found so far and see if anybody has other specific pieces of data they would like to see gathered. -- Inbound uptime -- I looked at a ~9 day period from April 7th to April 16th. During this time: * inbound was closed for 24.9587% of the total time * inbound was closed for 15.3068% of the total time due to bustage. * inbound was closed for 11.2059% of the total time due to infra. Notes: 1) bustage and infra were determined by grep -i on the data from treestatus.mozilla.org. 2) There is some overlap so bustage + infra != total. 3) I also weighted the downtime using checkins-per-hour histogram from joduinn's blog at [1], but this didn't have a significant impact: the total, bustage, and infra downtime percentages moved to 25.5392%, 15.7285%, and 11.3748% respectively. -- Backout changes -- Next I did an analysis of the changes that landed on inbound during that time period. The exact pushlog that I looked at (corresponding to the same April 7 - April 16 time period) is at [2]. I removed all of the merge changesets from this range, since I wanted to look at inbound in as much isolation as possible. In this range: * there were a total of 916 changesets * there were a total of 553 pushes * 74 of the 916 changesets (8.07%) were backout changesets * 116 of the 916 changesets (12.66%) were backed out * removing all backouts and changes backed out removed 114 pushes (20.6%) Of the 116 changesets that were backed out: * 37 belonged to single-changeset pushes * 65 belonged to multi-changeset pushes where the entire pushed was backed out * 14 belonged to multi-changeset pushes where the changesets were selectively backed out Of the 74 backout changesets: * 4 were for commit message problems * 25 were for build failures * 36 were for test failures * 5 were for leaks/talos regressions * 1 was for premature landing * 3 were for unknown reasons Notes: 1) There were actually 79 backouts, but I ignored 5 of them because they backed out changes that happened prior to the start of my range). 2) Additional changes at the end of my range may have been backed out, but the backouts were not in my range so I didn't include them in my analysis. 3) The 14 csets that were selectively backed out is interesting to me because it implies that somebody did some work to identify which changes in the push were bad, and this naturally means that there is room to save on doing that work. -- Merge conflicts -- I also wanted to determine how many of these changes conflicted with each other, and how far away the conflicting changes were. I got a partial result here but I need to do more analysis before I have numbers worth posting. -- Build farm resources -- Finally, I used a combination of gps' mozilla-build-analyzer tool [3] and some custom tools to determine how much machine time was spent on building all of these pushes and changes. I looked at all the build.json files [4] from the 6th of April to the 17th of April and pulled out all the jobs that corresponding to the push changesets in my range above. For this set of 553 changesets, there were 500 (exactly!) distinct builders. 111 of these had -pgo or _pgo in the name, and I excluded them. I created a 553x389 matrix with the remaining builders and filled in how much time was spent on each changeset for each builder (in case of multiple jobs, I added the times). Then I assumed that any empty field in the 553x389 matrix was a result of coalescing
Re: Some data on mozilla-inbound
On 4/23/13 6:35 PM, Kartikaya Gupta wrote: On 13-04-23 03:57 , Axel Hecht wrote: Do we know how many of these have been pushed to try, and passed/compiled what they'd fail later? I haven't looked at this. It would be useful to know but short of pulling patches and using some similarity heuristic or manually examining patches I can't think of a way to get this data. I expect some cost of regressions to come from merging/rebasing, and it'd be interesting to know how much of that you can see in the data window you looked at. This is something I did try to determine, by looking at the number of conflicts between patches in my data window. My algorithm was basically this: 1) Sync a tree to the last cset in the range 2) Iterate through each push backwards, skipping merges, backouts, and changes that are later backed out 3) For each of these pushes, try to qpush a backout of it. 4) If the attempted qpush fails, that means there is another change that landed since that one that there is a merge conflict with. The problem here is that the farther back you go the more likely it is that you will run into conflicting changes, because an increasing portion of the data window is checked for conflicts when really you probably only want to test some small number of changes (~30?). Using this approach I got 129 conflicts, and as expected, the rate at which I encountered conflicts went up as I went farther back. I didn't get around to trying the sliding window approach which I believe will give a more representative (and much lower) count. My code for doing this is in the bottom half of [1] if you (or anybody else) wants to give that a shot. I expect that only a part of our programmatic merge conflicts are actually version control merge conflicts. There are a lot of cases like modifications to supposedly internal properties in toolkit starting to get a new usecase in browser, a define changing or disappearing, etc. All those invalidate the testing of the patch that has been done to some extent, and don't involve modifications to the same lines of code, which is all that version control catches. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Nightly *very* crashy on OSX
Hi, I'm having a very crashy nightly, uptime below an hour, not really bound to a site. Might be https://bugzilla.mozilla.org/show_bug.cgi?id=864125, but I've experienced a bunch of crashes, all with pretty non-existing stack traces of no or one frame. bp-48ad9b29-145f-49ec-b282-5538f2130421 4/21/13 3:27 PM bp-bea9322a-ab85-4586-8f26-bfbcb2130421 4/21/13 2:45 PM bp-b3b43fa7-4c37-4f92-8d17-c82802130420 4/20/13 10:59 PM bp-7e7f70e9-85c9-4fd2-a2d6-31c892130420 4/20/13 8:34 PM bp-3faed1dd-98bb-4448-997b-db6f22130420 4/20/13 8:16 PM bp-5440caa0-7ebc-48e2-bd15-7fcf12130416 4/17/13 12:44 AM bp-3dbd9606-7d63-4a90-957a-98f772130416 4/17/13 12:32 AM bp-2b7ac91d-1110-4780-9370-89a372130416 4/17/13 12:31 AM Any ideas? Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Decoupling the build config via moz.build files (nuking all.js)
On 15.03.13 20:06, Benjamin Smedberg wrote: On 3/15/2013 2:33 PM, Gregory Szorc wrote: I /think/ our current spaghetti configuration is a historical artifact from using Makefile.in's to define the build config combined with the complexity required to do things right. Yes, I believe you are mostly correct. With moz.build files, we now have a mechanism to facilitate decentralized declaration of configuration and metadata. For example, instead of defining all preferences in /modules/libpref/src/init/all.js, we could have multiple prefs files each declared through multiple moz.build files. moz.build tree traversal would discover all relevant prefs files and the build code in /modules/libpref would simply concatenate that set of files together. No action outside of your module would be required! Note that we already started doing this using makefile-fu, see http://mxr.mozilla.org/mozilla-central/source/modules/libpref/src/Makefile.in#44 We should definitely continue. *Also* note that we actually have two different files: all.js is the defaults for the Mozilla platform, including Tbird/Seamonkey and all XULRunner apps. firefox.js is where Firefox-specific prefs and overrides typically should live. --BDS ... + firefox-l10n.js, for locale-specific settings. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Review of changes to Web compat-sensitive prefs in localizations
On 27.02.13 09:30, Henri Sivonen wrote: On Fri, Feb 22, 2013 at 8:03 PM, Axel Hecht l...@mozilla.com wrote: On 22.02.13 18:41, Henri Sivonen wrote: On Feb 22, 2013 5:30 PM, Axel Hecht l...@mozilla.com wrote: There's just no other way than post-mortem work. That's one of the reasons why we're not taking arbitrary changesets to ship to any audience beyond aurora and nightly, for beta and release, we got to have technical checks in place. Where should I file bugs to add checks to this set of checks? Not sure which checks you're talking about, so I can't really tell what you want. I meant checks like flagging attempts to go to beta with either of the following: * Detector pref not being blank except for a specific white list of particular values for the ru, uk, ja, ja-JP-Mac and zh-TW locales. * Fallback charset set to UTF-8 in any locale that doesn't already have it set to UTF-8. I'm doing a source-based review, which at least catches regressions to those settings. And I think we're doing charset detector settings wrong. Let me see if I get right what we're doing: - most content should be labeled for charset - if not, let's see if we can guess the encoding -- if we assume the language of the content, we can guess better -- many languages really only have one option -- ru, uk, ja, zh-TW do have options, so we use a charset detector Now, I don't think it's right to use the UI language to guess content language. We have a list of user-preferred languages (with good defaults based on UI language). We should go through that list, and pick charsets to try for unlabeled content from there. That's rather orthogonal to what you're currently trying to do, but it's also indicating to me that we should remove all of those settings from intl.properties, and just leave accept-lang, and deduce the rest. You also mentioned in the bug that you didn't get the OK to use telemetry to gather further data. I think if we just collect the data about the charset optimization and how good it's doing, we should be OK. I.e., at the point where the locale doesn't matter, but just cp-1252 etc, the entropy goes up a good deal. In particular for small locales. I'd argue that this might even make sense to be part of health report. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Review of changes to Web compat-sensitive prefs in localizations
On 22.02.13 15:37, Henri Sivonen wrote: I've been finding and, to a lesser extent, reporting and writing patches for bugs where a localization sets the fallback encoding to a value that doesn't suit the purpose of the fallback. In some cases, there such bogosity in the intl.properties file (e.g. translation of the word windows as part of a charset label) that I suspect that changes to intl.properties have been landing without review. I propose we adopt a rule that says that localizations need review from the HTML parser module owner (i.e. me) to change the values of preferences that modify the behavior of the HTML parser. (In practice, this means the localizable properties intl.charset.default and intl.charset.detector.) Opinions? I don't think that .platform is the right group to discuss policies for l10n, tbh. Anyway, I don't think that it requires your review. For one, these rules just don't work in practice. We're facing the very same problem with search engines. There's just no other way than post-mortem work. That's one of the reasons why we're not taking arbitrary changesets to ship to any audience beyond aurora and nightly, for beta and release, we got to have technical checks in place. I usually catch regressions to intl.properties when reviewing requests for updates to those changesets. That said, I don't know what intl.charset.detector should be set to, aside from nothing. Looking at your patch, the comment doesn't make that clearer, too, I'll follow up there. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Review of changes to Web compat-sensitive prefs in localizations
On 22.02.13 20:02, L. David Baron wrote: On Friday 2013-02-22 16:37 +0200, Henri Sivonen wrote: I've been finding and, to a lesser extent, reporting and writing patches for bugs where a localization sets the fallback encoding to a value that doesn't suit the purpose of the fallback. In some cases, there such bogosity in the intl.properties file (e.g. translation of the word windows as part of a charset label) that I suspect that changes to intl.properties have been landing without review. It might not be a bad idea to have a better explanation in http://mxr.mozilla.org/mozilla-central/source/toolkit/locales/en-US/chrome/global/intl.properties of why one would want to change intl.charset.default and intl.charset.detector, explaining clearly that they should only be set to interesting values to deal with a substantial body of legacy content that requires those values, and then saying what they should be in the absence of such legacy content (the latter should clearly be empty; I'm not sure whether the former should be UTF-8 or ISO-8859-1, but we should have a consistent policy). That said, I don't actually know whether the tools localizers use to do localization lead them to read the text. The reality is that I suspect it may be important for you to do occasional audits of these values; it could also be valuable to have a tool that exposes all of them in a single place (perhaps even a place with history, like an automatically-generated wiki page). -David Henri filed https://bugzilla.mozilla.org/show_bug.cgi?id=844042 before posting here (or at least around the same time). Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Sparse localizations of toolkit and the chrome registry's getSelectedLocale()
On 12.02.13 22:12, Axel Hecht wrote: On 12.02.13 20:27, Benjamin Smedberg wrote: On 2/12/2013 12:41 PM, Axel Hecht wrote: Hi Benjamin, Dave, for mobile (and fxos) we're looking into doing sparse localizations of toolkit. The work on the mobile side is bug 792077. The current attempt is to leave the 'global' and other toolkit packages untranslated, and use chrome overrides to supply a few localized files, as needed by fx-android or -os. That's all kinda nice and shiny, but it horribly breaks getSelectedLocale(package), as that returns 'en-US' for 'global' and friends. Breaks all kinds of url reporters and rtl detection. Why does getSelectedLocale(package) matter? Who uses that API to make important decisions? And can we just change the callers to do something different? --BDS http://mxr.mozilla.org/mozilla-central/ident?i=getSelectedLocalefilter= is the call-sites. There's a bunch of .getSelectedLocale(global) to determine the currently active locale, independent of OS settings on linux. Some more detail: We'd need some package to ask here. We might get away abstracting this into an API on the chrome registry, and for apps with sparse l10n, have an overload pref that determines the package to look at. I actually started a patch on this one, but ... The other half of it is http://mxr.mozilla.org/mozilla-central/ident?i=IsLocaleRTL, which is a tad harder to resolve. That depends all over the place. ... realized that I have no idea on how to fix these. Benjamin? Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform