Is it OK to make allocations that intentionally aren't freed? (was: Re: Is Big5 form submission fast enough?)

[Henri Sivonen]

On Tue, May 16, 2017 at 7:03 AM, Tim Guan-tin Chien
 wrote:
> According to Alexa top 100 Taiwan sites and quick spot checks, I can only
> see the following two sites encoded in Big5:
>
> http://www.ruten.com.tw/
> https://www.momoshop.com.tw/
>
> Both are shopping sites (eBay-like and Amazon-like) so you get the idea how
> forms are used there.

Thank you. It seems to me that encoder performance doesn't really
matter for sites like these, since the number of characters one would
enter in the search field at a time is very small.

> Mike reminded me to check the Tax filing website: http://www.tax.nat.gov.tw/
> .Yes, it's unfortunately also in Big5.

I guess I'm not going to try filing taxes there for testing. :-)

- -

One option I've been thinking about is computing an encode
acceleration table for JIS X 0208 on the first attempt to encode a CJK
Unified Ideograph in any of Shift_JIS, EUC-JP or ISO-2022-JP, for GBK
on the first attempt to encode a CJK Unified Ideograph in either GBK
or gb18030, and for Big5 on the first attempt to encode a CJK Unified
Ideograph in Big5.

Each of the three tables would then remain allocated through to the
termination of the process.

This would have the advantage of not bloating our binary footprint
with data that can be computed from other data in the binary while
still making legacy Chinese and Japanese encode fast without a setup
cost for each encoder instance.

The downsides would be that the memory for the tables wouldn't be
reclaimed if the tables aren't needed anymore (the browser can't
predict the future) and executions where any of the tables has been
created wouldn't be valgrind-clean. Also, in the multi-process world,
the tables would be recomputed per-process. OTOH, if we shut down
rendered processes from time to time, it would work as a coarse
mechanism to reclaim the memory is case Japanese or Chinese legacy
encode is a relatively isolated event in the user's browsing pattern.

Creating a mechanism for the encoding library to become aware of
application shutdown just in order to be valgrind-clean would be
messy, though. (Currently, we have shutdown bugs where uconv gets used
after we've told it can shut down. I'd really want to avoid
re-introducing that class of bugs with encoding_rs.)

Is it OK to create allocations that are intentionally never freed
(i.e. process termination is what "frees" them)? Is valgrind's message
suppression mechanism granular enough to suppress three allocations
from a particular Rust crate statically linked into libxul?

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Quantum Flow Engineering Newsletter #10

[Ehsan Akhgari]

Hi everyone,

Let's start this week's updates with looking at the ongoing efforts to
improve the usefulness of the background hang reports data.  With Ben
Miroglio's help, we confirmed
 that
we aren't blowing up telemetry ping sizes yet by sending native stack
traces for BHR hangs, and as a result we can now capture a deeper call
stack depth , which
means the resulting data will be easier to analyze.  Doug Thayer has also
been hard at work  at
creating a new BHR dashboard based on the perf-html UI.  You can see a
sneak peak here , but do note that
this is work in progress!  The raw BHR data
 is still available
for your inspection.

Kannan Vijayan has been working on adding some low level instrumentation
 to SpiderMonkey in
order to get some detailed information on the relative runtime costs of
various builtin intrinsic operations inside the JS engine in various
workloads using the rdtsc instruction on Windows.  He now has a working
setup that allows him to take a real world JS workload and get some
detailed data on what builtin intrinsics were the most costly in that
workload.  This is extremely valuable because it allows us to focus our
optimization efforts on these builtins where the most gains are to be
achieved first.  He already has some initial results
 of running this tool
on the Speedometer benchmark and on a general browsing workload and some
optimization work has already started
 to happen
.

Dominik Strohmeier has been helping with running startup measurements

on the reference Acer machine to track the progress of the ongoing startup
improvements using an HDMI video capture card.  For these measurements, we
are tracking two numbers, one is the first paint times (the time at which
we paint the first frame from the browser window) and the other is the hero
element time (the time at which we paint the "hero element" which is the
search box in about:home in this case.)  The baseline build here is the
Nightly of Apr 1st as a date before active work on startup optimizations
started.  At that time, our median first paint time was 1232.84ms (with a
standard deviation of 16.58ms) and our hero element time was
1849.26ms (with a standard deviation of 28.58ms).  On the Nightly of May
18, our first paint time is 849.66ms (with a standard deviation of 11.78ms)
and our hero element time is 1616.02ms (with a standard deviation of
24.59ms).

Next week we're going to have a small work week with some people from the
DOM, JS, Layout, Graphics and Perf teams here in Toronto.  I expect to be
fully busy at the work week, so you should expect the next issue of this
newsletter in two weeks!  With that, it is time to acknowledge the hard
work of those who helped make Firefox faster this past week.  I hope I'm
not dropping any names by accident!

   - Olli Pettay wrote a patch to have us coalesce scroll wheel events more
   aggressively , in
   the hopes of improving scrolling performance on (at least) Google
   Spreadsheets
   - Dão Gottwald got rid of some nasty layout flushes in our tab strip code
    when adding and
   removing tabs
   - Kris Maglione made the content process also use the script precompiler
   in order to speed up the startup by compiling scripts on a background thread
   .  This resulted
   in some decent improvements
    in Talos
   sessionrestore tests.
   - Kris also got rid of some directory scans that the Add-ons Manager was
   doing on start-up ,
   which should result in improved start-up times.
   - Mike Conley suppressed the window opening animation for the first
   window , buying us
   some more time during startup (about 270ms on the reference hardware
   

   !)
   - Florian Quèze switched many uses of Task.jsm / yield to async
   functions / await instead
   , which allows us
   to avoid cross-compartment wrapper overhead in certain situations. This was
   a huge undertaking - big kudos to Florian!
   - Dave Townsend 

Re: Volunteer maintainer wanted: mac x86

[Nicholas Nethercote]

On Wed, May 17, 2017 at 5:28 PM, Nicholas Nethercote  wrote:

> https://developer.mozilla.org/en/docs/Supported_build_configurations
> still lists x86/Mac as a tier 1 platform.
>
> Should it be moved to tier 3, or removed altogether?
>

If anyone was wondering... I was about to remove it altogether from that
page, but I see that Ralph got in ahead of me. Thanks, Ralph!

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Avoiding jank in async functions/promises?

[Mark Hammond]

On 5/18/17 12:03 PM, Boris Zbarsky wrote:

On 5/17/17 9:22 PM, Mark Hammond wrote:

I'm wondering if there are any ideas about how to solve this optimally?


I assume 
https://w3c.github.io/requestidlecallback/#the-requestidlecallback-method doesn't 
have quite the right semantics here?  That would let you run when the 
browser is idle, and give you some idea of how long you can run for 
before you should yield.


I didn't quite expect this to work, but by abusing rIC I can almost make 
something work - I just have the callback stash the IdleDeadline object 
and return immediately, but continue to refer to it anyway. eg:


let idleChecker = {
  resolved: Promise.resolve(),
  deadline: null,
  promiseIdle() {
if (this.deadline && this.deadline.timeRemaining() > 0) {
  return this.resolved;
}
this.deadline = null
return new Promise(resolve => {
  window.requestIdleCallback(deadline => {
this.deadline = deadline;
resolve();
  });
})
  }
}

async function janky() {
  let start = Date.now();
  for (let i = 0; i < 1000; i++) {
await Promise.resolve();
await idleChecker.promiseIdle();
  }
  console.log("took", Date.now() - start)
}
janky().then(() => console.log("done"));

I 1/2 expect this to defeat the intent/implementation of rIC, but it 
does work, causing ~2x-5x slowdown of the loop. I wonder if this is 
worth experimenting with some more?


Mark


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Changing our thread APIs for Quantum DOM scheduling

[Bill McCloskey]

Hi everyone,

One of the challenges of the Quantum DOM project is that we will soon have
multiple "main" threads [1]. These will be real OS threads, but only one of
them will be allowed to run code at any given time. We will switch between
them at well-defined points (currently just the JS interrupt callback).
This cooperative scheduling will make it much easier to keep our global
state consistent. However, having multiple "main" threads is likely to
cause confusion.

To simplify things, we considered trying to make these multiple threads
"look" like a single main thread at the API level, but it's really hard to
hide something like that. So, instead, we're going to be transitioning to
APIs that try to avoid exposing threads at all. This post will summarize
that effort. You can find more details in this Google doc:

https://docs.google.com/document/d/1MZhF1zB5_dk12WRiq4bpmNZUJWmsIt9OTpFUWAlmMyY/edit?usp=sharing

[Note: I'd like this thread (and the Google doc) to be for discussing
threading APIs. If you have more general questions about the project,
please contact me personally.]

The main API change is that we're going to make it a lot harder to get hold
of an nsIThread for the main thread. Instead, we want people to work with
event targets (nsIEventTarget). The advantage of event targets is that all
the main threads will share a single event loop, and therefore a single
nsIEventTarget. So code that once expected a single main thread can now
expect a single main event target.

The functions NS_GetMainThread, NS_GetCurrentThread, and
do_Get{Main,Current}Thread will be deprecated. In their place, we'll
provide mozilla::Get{Main,Current}ThreadEventTarget. These functions will
return an event target instead of a thread.

More details:

NS_IsMainThread: This function will remain pretty much the same. It will
return true on any one of the main threads and false elsewhere.

Dispatching runnables: NS_DispatchToMainThread will still work, and you
will still be able to dispatch using Get{Main,Current}ThreadEventTarget.
>From JS, we want people to use Services.tm.dispatchToMainThread.

Thread-safety assertions: Code that used PR_GetCurrentThread for thread
safety assertions will be converted to use NS_ASSERT_OWNINGTHREAD, which
will allow code from different main threads to touch the same object.
PR_GetCurrentThread will be deprecated. If you really want to get the
current PRThread*, you can use GetCurrentPhysicalThread, which will return
a different value for each main thread.

Code that uses NS_GetCurrentThread for thread safety assertions will be
converted to use nsIEventTarget::IsOnCurrentThread. The main thread event
target will return true from IsOnCurrentThread if you're on any of the main
threads.

Nested event loop spinning: In the future, we want people to use
SpinEventLoopUntil to spin a nested event loop. It will do the right thing
when called on any of the main threads. We'll provide a similar facility to
JS consumers.

Bugs:

Bug 1359903 converted some of our PR_GetCurrentThread assertions to use
NS_ASSERT_OWNINGTHREAD.

Bug 1361561 added GetCurrentPhysicalThread and GetCurrentVirtualThread.
These functions both return a PRThread*. The latter one returns the same
value for all the main threads. It should only be used for assertion
checking.

Bug 1361164 will add the Get{Current,Main}ThreadEventTarget functions.

Bug 1365096 is a metabug to convert all uses of NS_Get{Current,Main}Thread
to use event targets instead. Bug 1365097 is the bug for converting DOM
code.

[1] https://billmccloskey.wordpress.com/2016/10/27/mozillas-quantum-project/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Removing Jemalloc 4

[jasone]

This is a disappointing outcome.  You put a huge amount of work into porting 
and testing jemalloc, and more recently I've done a lot of work to improve 
reliability especially on Windows, primarily with Firefox in mind.  Over the 
past few months we've fixed some long-standing performance issues that were 
peculiar to Windows. Even more recently, Qi Wang, David Goldblatt and I have 
done a lot of optimization in anticipation of the jemalloc 5.0.0 release, which 
is shaping up to be really solid.  It isn't clear to me whether a final 
detailed Firefox+jemalloc[45] performance analysis happened after those fixes 
and optimizations; the bits of analysis I see evidence of in bugzilla suggest 
that some simple issues such as the behavior of decay-based purging may not 
have been taken into consideration.  

My disappointment aside, thank you for all your contributions to jemalloc.  You 
influenced its development in ways that continue to have a positive impact, 
especially as related to testing.  Although all the jemalloc team members are 
currently employed by Facebook, we are developing jemalloc completely in the 
open in large part so that it is feasible to collaborate with other 
organizations and robustly handle diverse workloads.  I wish Firefox would 
continue to be part of our target.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Gecko Profiler Win64 stability

[Mike Conley]

Tremendous work here. Thank you so much, dmajor!

On 2017-05-18 3:04 PM, David Major wrote:
> Hi,
> 
> The Gecko Profiler used to be notoriously unstable on 64-bit Windows.
> (If you're curious: it's mostly because the unwinding rules for Win64
> require the system libraries to do a lot of synchronization, which our
> stack walker would often call in ways that could deadlock.)
> 
> Ever since the last Quantum Flow work week, I've been working on fixing
> these issues. As of this week's nightlies, all of the hangs that have
> been reported to me are resolved fixed. There might be some rare corner
> cases still remaining, but I'm confident that the most frequent problems
> have been addressed.
> 
> So my ask is: if you've previously been deterred from using the profiler
> on Win64 builds, please give it another try! Flag me on any issues you
> find and/or mark them blocking bug 1366030. I will treat these bugs with
> high priority since I want to keep Quantum activities unblocked.
> 
> Thanks!
> 
> 
> ___
> firefox-dev mailing list
> firefox-...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
> 
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Gecko Profiler Win64 stability

[David Major]

Hi,

The Gecko Profiler used to be notoriously unstable on 64-bit Windows. (If
you're curious: it's mostly because the unwinding rules for Win64 require
the system libraries to do a lot of synchronization, which our stack walker
would often call in ways that could deadlock.)

Ever since the last Quantum Flow work week, I've been working on fixing
these issues. As of this week's nightlies, all of the hangs that have been
reported to me are resolved fixed. There might be some rare corner cases
still remaining, but I'm confident that the most frequent problems have
been addressed.

So my ask is: if you've previously been deterred from using the profiler on
Win64 builds, please give it another try! Flag me on any issues you find
and/or mark them blocking bug 1366030. I will treat these bugs with high
priority since I want to keep Quantum activities unblocked.

Thanks!
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Avoiding jank in async functions/promises?

[Domenic Denicola]

On Thursday, May 18, 2017 at 4:34:37 AM UTC-4, smaug wrote:
> FWIW, I just yesterday suggested in #whatwg that the platform should have 
> something like IdlePromise or AsyncPromise.
> And there is the related spec bug 
> https://github.com/whatwg/html/issues/512#issuecomment-171498578

In my opinion there's no need for a whole new promise subclass, just make 
requestIdleCallback() return a promise when there's no argument passed.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Switching to async/await from Task.jsm/yield

[Kris Maglione]

On Mon, May 15, 2017 at 10:14:10AM -0700, Sam Foster wrote:

I think this deserves a round of applause. Its a large undertaking with
measurable perf wins in areas with strong SEP(1) fields


Do you have measurements? It seems likely to be a perf win, but 
I haven't seen any metrics to confirm what effects it's had.



On Wed, May 10, 2017 at 7:09 PM, Florian Quèze  wrote:


On Mon, Mar 27, 2017 at 9:42 PM, Dave Townsend 
wrote:

> lets go ahead with this and avoid using Task.jsm in the future.

Patches to remove Task.jsm usage from browser/ and toolkit/ are now
available in https://bugzilla.mozilla.org/show_bug.cgi?id=1353542
along with the scripts that generated them.

I currently expect to land them on Friday morning. Given the size of
the patches (2040 files changed, more than 35k lines changed), this
landing will likely happen with a tree closure, to avoid merging
conflicts between inbound and autoland.

If you have WIP front-end patches that are almost ready to land, I
would suggest landing them soon to avoid bitrot.

Florian

--
Florian Quèze
___
firefox-dev mailing list
firefox-...@mozilla.org
https://mail.mozilla.org/listinfo/firefox-dev




___
firefox-dev mailing list
firefox-...@mozilla.org
https://mail.mozilla.org/listinfo/firefox-dev



--
Kris Maglione
Senior Firefox Add-ons Engineer
Mozilla Corporation

The cardinal doctrine of a fanatic's creed is that his enemies are the
enemies of God.
--Andrew Dickson White

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: IDNA processing

[Gervase Markham]

On 18/05/17 14:14, Anne van Kesteren wrote:
> That's fairly non-specific, unless you really mean that you don't want
> "A" lowercased.

Well, yes, as you note, with UTS#46 or whatever it is.

> I don't think it's that big, there's plenty of other things disallowed
> that we should always be able to find something, if it comes to that.

Then I think whatever you decide about how to deal with ASCII fast paths
is fine. :-)

Gerv
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: IDNA processing

[Anne van Kesteren]

On Mon, May 15, 2017 at 3:38 PM, Gervase Markham  wrote:
> Well, you generally know my opinion :-) IDNA 2008 non-transitional.

That's fairly non-specific, unless you really mean that you don't want
"A" lowercased.


> ISTM that the 3rd/4th placed hyphens were banned so the domain name
> system had an extension mechanism, and that was used for IDNA (xn--). If
> we allow domains of that form, we no longer have that extension
> mechanism. The question is, how big a loss is that?

I don't think it's that big, there's plenty of other things disallowed
that we should always be able to find something, if it comes to that.


I ended up doing some more work on this and created a bunch of tests
and filed various browser bugs. Not applying ToASCII to ASCII-only
input is its own can of worms, due to input such as "xn--a" then
round-tripping poorly. https://github.com/whatwg/url/issues/267 has
the relevant details and pointers.


-- 
https://annevankesteren.nl/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Avoiding jank in async functions/promises?

[smaug]

FWIW, I just yesterday suggested in #whatwg that the platform should have 
something like IdlePromise or AsyncPromise.
And there is the related spec bug 
https://github.com/whatwg/html/issues/512#issuecomment-171498578


On 05/18/2017 04:22 AM, Mark Hammond wrote:

Given our recent performance push, I thought I'd bring this topic up again.

When writing loops using async functions in browser code, it's still very easy 
to cause jank. For example, a trivial function:


async function janky() {
  for (let i = 0; i < 10; i++) {
await Promise.resolve();
  }
}

janky().then(() => console.log("done"));


will cause the browser to hang. While that's not considered a bug in the 
implementation of the promise scheduler, and might not be particularly
surprising in that trivial example, lots of real-world code can still hit this 
case - which both isn't obvious from casual inspection, and even if it
was, doesn't have an obvious solution.

Concretely, we struck this a couple of years ago in bug 1186714 - creating a 
backup of all bookmarks ends up looking alot like the loop above. In
addition, the Sync team is moving away from nested event loops towards 
promises, and our work to date is hitting this problem.

In bug 1186714, we solved the problem by inserting code into the loop that 
looks like:


   if (i % 50 == 0) {
 await new Promise(resolve => {
   Services.tm.dispatchToMainThread(resolve);
 });
   }


http://searchfox.org/mozilla-central/rev/f55349994fdac101d121b11dac769f3f17fbec4b/toolkit/components/places/PlacesUtils.jsm#2022

so we explicitly yield to the event loop every 50 iterations. However, this 
isn't optimal as the 50 is obviously a magic number, determined by
experimentation on a couple of machines - when running on low spec hardware, 
this loop is almost certainly still janky. If we try and err on the side
of caution (eg, yielding every iteration) we see the wall time of the loop take 
a massive hit (around twice as long in that bug).

I'm wondering if there are any ideas about how to solve this optimally? Naively, it seems 
that the (broadest sense of the term) "platform" might be
able to help here using it's knowledge of the event-loop state - eg, a function that 
indicates "are we about to starve the event loop and become
janky?", or possibly even the whole hog (ie, "please yield to the event loop if 
you think now is a good time, otherwise just hand back a pre-resolved
promise").

Or maybe there are simpler options I've overlooked?

Thanks,

Mark


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform