Scope of XML parser rewrite?
In reference to: https://twitter.com/nnethercote/status/866792097101238272 Is the rewrite meant to replace expat only or also some of our old code on both above and below expat? Back in 2011, I wrote a plan for rewriting the code around expat without rewriting expat itself: https://wiki.mozilla.org/Platform/XML_Rewrite I've had higher-priority stuff to do ever since... (The above plan talks about pushing UTF-16 to the XML parser and having deep C++ namespaces. Any project starting this year should make the new parser use UTF-8 internally for cache-friendliness and use less deep C++ namespaces.) Also, I think the decision of which XML version to support should be a deliberate decision and not an accident. I think the reasonable choices are XML 1.0 4th edition (not rocking the boat) and reviving XML5 (original discussion: https://annevankesteren.nl/2007/10/xml5 , latest draft: https://ygg01.github.io/xml5_draft/). XML 1.1 is dead. XML 1.0 5th edition tried to have the XML 1.0 cake and eat the XML 1.1 cake too and expanded the set of documents that parser doesn't reject. Any of the newly well-forming documents would be incompatible with 4th ed. and earlier parsers, which would be a break from universal XML interop. I think it doesn't make sense to relax XML only a bit. If XML is to be relaxed (breaking interop in the sense of starting to accept docs that old browsers would show the Yellow Screen of Death on), we should go all the way (i.e. XML5). Notably, it looks like Servo already has an XML5 parser written in Rust: https://github.com/servo/html5ever/tree/master/xml5ever The tweets weren't clear about whether xml5ever had been considered, but https://twitter.com/eroc/status/866808814959378434 looks like it's talking about writing a new one. It seems like integrating xml5ever (as opposed to another XML parser written in Rust) into Gecko would give some insight into how big a deal it would be to replace Gecko's HTML parser with html5ever (although due to document.write(), HTML is always a bigger deal integration-wise than XML). (If the outcome here is to do XML5, we should make sure the spec is polished enough at the WHATWG in order not to a unilateral thing in relative secret.) -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Start logging at runtime (Firefox 52)
Which platform you're using? For windows it seems to be solved by https://bugzilla.mozilla.org/show_bug.cgi?id=1320458, however other platforms are not fixed yet. Best Regards, Shih-Chiang Chien Mozilla Taiwan On Tue, May 23, 2017 at 11:59 AM, Chris Pearce wrote: > On Sunday, November 27, 2016 at 5:59:27 AM UTC+13, Valentin Gosu wrote: > > Hi everyone, > > > > (I meant to send this mail a few weeks ago but forgot it in my Drafts > > folder.) > > > > With the landing of Bug 1303762 (Firefox 52), we now have a way for users > > to enable logging without restarting the browser, and without having to > > know what an environment variable is. > > > > We've added a new Logging section to about:networking. When a user > > encounters a bug, all they have to do is open that page, click on "Start > > Logging", reproduce the bug, then click on "Stop Logging" and upload the > > logs. The buttons will be disabled if the MOZ_LOG_FILE or MOZ_LOG env > > variables have been defined. > > The log modules are automatically set to the most common networking > > modules, but you may instruct the bug reporters to change them - just > tell > > them the string. > > > > This is very useful for bugs that are harder to reproduce once you > restart > > the browser. > > > > There are a bunch of improvements that we could make to the UI, so please > > feel free to send me your feedback and patches. Many thanks to Honza > > Bambas, Eric Rahm, Jared Wein, Patrick McManus and all others that helped > > with the implementation and review of this bug and its dependencies. > > > > https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/ > HTTP_logging?document_saved=true#Using_aboutnetworking > > This seems super handy, but I tried it out and it seems to only affect the > parent process, and not the sub-processes of Firefox? > > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Start logging at runtime (Firefox 52)
On Sunday, November 27, 2016 at 5:59:27 AM UTC+13, Valentin Gosu wrote: > Hi everyone, > > (I meant to send this mail a few weeks ago but forgot it in my Drafts > folder.) > > With the landing of Bug 1303762 (Firefox 52), we now have a way for users > to enable logging without restarting the browser, and without having to > know what an environment variable is. > > We've added a new Logging section to about:networking. When a user > encounters a bug, all they have to do is open that page, click on "Start > Logging", reproduce the bug, then click on "Stop Logging" and upload the > logs. The buttons will be disabled if the MOZ_LOG_FILE or MOZ_LOG env > variables have been defined. > The log modules are automatically set to the most common networking > modules, but you may instruct the bug reporters to change them - just tell > them the string. > > This is very useful for bugs that are harder to reproduce once you restart > the browser. > > There are a bunch of improvements that we could make to the UI, so please > feel free to send me your feedback and patches. Many thanks to Honza > Bambas, Eric Rahm, Jared Wein, Patrick McManus and all others that helped > with the implementation and review of this bug and its dependencies. > > https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging?document_saved=true#Using_aboutnetworking This seems super handy, but I tried it out and it seems to only affect the parent process, and not the sub-processes of Firefox? ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Is it OK to make allocations that intentionally aren't freed?
On Mon, May 22, 2017 at 4:45 PM, Karl Tomlinson wrote: > Andrew McCreight writes: > > > On Fri, May 19, 2017 at 7:38 PM, Nicholas Nethercote < > n.netherc...@gmail.com > >> wrote: > > > >> There's also a pre-processor constant that we define in > Valgrind/ASAN/etc. > >> builds that you can check in order to free more stuff than you otherwise > >> would. But I can't for the life of me remember what it's called :( > >> > > > > NS_FREE_PERMANENT_DATA. > > I understand the need to clean up objects for > NS_BUILD_REFCNT_LOGGING, but how important is this for objects not > tracked with NS_BUILD_REFCNT_LOGGING, but only MOZ_VALGRIND and/or > MOZ_ASAN? > > I thought I'd seen enough unfreed allocations that can be > referenced from a root in static storage, that I assumed that > valgrind and ASAN ignored these allocations in their leak report. > That is correct for LSan. I don't know how Valgrind reports leaks. Is the inclusion of MOZ_VALGRIND and MOZ_ASAN in the value of > NS_FREE_PERMANENT_DATA merely to make leak detection easier for > valgrind and ASAN, or will such allocations be reported as leaks > by these tools? > It will let you clean up things that are not referred to by static storage, and change behavior in other ways that are needed for leak checking. For instance, we do our shutdown CCs when it is defined, and disables the timeout for shutting down a child process. ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Is it OK to make allocations that intentionally aren't freed?
Andrew McCreight writes: > On Fri, May 19, 2017 at 7:38 PM, Nicholas Nethercote > wrote: > >> There's also a pre-processor constant that we define in Valgrind/ASAN/etc. >> builds that you can check in order to free more stuff than you otherwise >> would. But I can't for the life of me remember what it's called :( >> > > NS_FREE_PERMANENT_DATA. I understand the need to clean up objects for NS_BUILD_REFCNT_LOGGING, but how important is this for objects not tracked with NS_BUILD_REFCNT_LOGGING, but only MOZ_VALGRIND and/or MOZ_ASAN? I thought I'd seen enough unfreed allocations that can be referenced from a root in static storage, that I assumed that valgrind and ASAN ignored these allocations in their leak report. Is the inclusion of MOZ_VALGRIND and MOZ_ASAN in the value of NS_FREE_PERMANENT_DATA merely to make leak detection easier for valgrind and ASAN, or will such allocations be reported as leaks by these tools? ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Improving visibility of compiler warnings
On Sat, 20 May 2017 14:59:11 -0400, Eric Rescorla wrote: > On Sat, May 20, 2017 at 1:16 PM, Kris Maglione > wrote: > >> On Sat, May 20, 2017 at 08:36:13PM +1000, Martin Thomson wrote: >> >>> Hmm, these are all -Wsign-compare issues bar one, which is fixed >>> upstream. We have an open bug tracking the warnings >>> (https://bugzilla.mozilla.org/show_bug.cgi?id=1307958 and specifically >>> https://bugzilla.mozilla.org/show_bug.cgi?id=1212199 for NSS). >>> >>> NSS is built with -Werror separately, but disables errors on >>> -Wsign-compare. Disabling those warnings for a Firefox build of NSS >>> wouldn't be so bad now that we share gyp config. Based on a recent >>> build, that's 139 messages (add 36 if you want to add nspr). >>> >> >> Is there a particularly good reason that NSS needs to disable >> -Wsign-compare? That seems like a footgun waiting to happen, especially in >> crypto code. > > > Like many other pieces of old code, there are a lot of things that trigger > compiler warnings which we have been progressively removing, but we > haven't removed these yet. Ideally we would have some tooling that > would distinguish new warnings from old warnings, but absent that, > until we fix all the existing warnings it's either disable -Werror or > disable this particular warning. It's not something we're particularly > happy about. -Wno-error=sign-compare is also an option, but a wall of sign-compare warnings may not be helpful in general. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Avoiding jank in async functions/promises?
On Sun, May 21, 2017, at 09:29 PM, Mark Hammond wrote: > As I mentioned at the start of the thread, in one concrete example we > had code already written that we identified being janky - > http://searchfox.org/mozilla-central/rev/f55349994fdac101d121b11dac769f3f17fbec4b/toolkit/components/places/PlacesUtils.jsm#2022 Is this a scenario where we could move this intensive logic off the (parent process) main thread by fulfilling the dream of the "SQLite interface for Workers" bug[1] by using WebIDL instead of js-ctypes to let the Sqlite.jsm abstraction operate on ChromeWorkers? The only XPConnect leakage on the Sqlite.jsm API surface is mozIStorageStatementRow[2], and although it's a bit unwieldy in terms of method count, we never exposed any of the nsIXPCScriptable magic on it that we did on statements. (And thankfully SQLite.jsm neither uses or otherwise exposes the API.) It wouldn't be a trivial undertaking, but it's also not impossible either. And if Sync is chewing up a lot of main thread time both directly (processing) and indirectly (generating lots of of garbage that lengthens parent-process main-thread GC's), it may be worth considering rather than trying to optimize the time-slicing of Sync. This does, of course, assume that Sync can do meaningful work without access to XPConnect and that there aren't major gotchas in coordinating with Places' normal operation. Note: I'm talking exclusively about using the existing asynchronous Sqlite.jsm API on top of the existing async mozStorage API usage. Andrew 1: https://bugzilla.mozilla.org/show_bug.cgi?id=853438 2: https://searchfox.org/mozilla-central/source/storage/mozIStorageRow.idl subclasses https://searchfox.org/mozilla-central/source/storage/mozIStorageValueArray.idl ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Mozilla Charset Detectors
On Mon, May 22, 2017 at 12:13 PM, Gabriel Sandor wrote: > I recently came across the Mozilla Charset Detectors tool, at > https://www-archive.mozilla.org/projects/intl/chardet.html. I'm working on > a C# project where I could use a port of this library (e.g. > https://github.com/errepi/ude) for advanced charset detection. It's somewhat unfortunate that chardet got ported over to languages like Python and C# with its shortcomings. The main shortcoming is that despite the name saying "universal", the detector was rather arbitrary in what it detected and what it didn't. Why Hebrew and Thai but not Arabic or Vietnamese? Why have a Hungarian-specific frequency model (that didn't actually work) but no models for e.g. Polish and Czech from the same legacy encoding family? The remaining detector bits in Firefox are for Japanese, Russian and Ukrainian only, and I strongly suspect that the Russian and Ukrainian detectors are doing more harm than good. > I'm not sure however if this tool is deprecated or not, and still > recommended by Mozilla for use in modern applications. The tool page is > archived and most of the links are dead, while the code seems to be at > least 7-8 years old. Could you please tell me what's the status of this > tool and whether I should use it in my project or look for something else? I recommend not using it. (I removed most of it from Firefox.) I recommend avoiding heuristic detection unless your project absolutely can't do without. If you *really* need a detector, ICU and https://github.com/google/compact_enc_det/ might be worth looking at, though this shouldn't be read as an endorsement of either. With both ICU and https://github.com/google/compact_enc_det/ , watch out for the detector's possible guess space containing very rarely used encodings that you really don't want content detected as by mistake. -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Mozilla Charset Detectors
On 22/05/2017 10:13, Gabriel Sandor wrote: Greetings, I recently came across the Mozilla Charset Detectors tool, at https://www-archive.mozilla.org/projects/intl/chardet.html. I'm working on a C# project where I could use a port of this library (e.g. https://github.com/errepi/ude) for advanced charset detection. I'm not sure however if this tool is deprecated or not, and still recommended by Mozilla for use in modern applications. The tool page is archived and most of the links are dead, while the code seems to be at least 7-8 years old. Could you please tell me what's the status of this tool and whether I should use it in my project or look for something else? I'd suggest looking at ICU, for a modern, actively-maintained library that can probably help you: http://userguide.icu-project.org/conversion/detection Or there's also https://github.com/google/compact_enc_det (as mentioned in the ICU doc). JK ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Mozilla Charset Detectors
Greetings, I recently came across the Mozilla Charset Detectors tool, at https://www-archive.mozilla.org/projects/intl/chardet.html. I'm working on a C# project where I could use a port of this library (e.g. https://github.com/errepi/ude) for advanced charset detection. I'm not sure however if this tool is deprecated or not, and still recommended by Mozilla for use in modern applications. The tool page is archived and most of the links are dead, while the code seems to be at least 7-8 years old. Could you please tell me what's the status of this tool and whether I should use it in my project or look for something else? Thanks in advance. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
