RE: Breakdown of Firefox full installer
-Original Message- From: dev-platform [mailto:dev-platform- bounces+rstrong=mozilla@lists.mozilla.org] On Behalf Of Robert Kaiser Sent: Saturday, November 1, 2014 6:47 PM To: dev-platform@lists.mozilla.org Subject: Re: Breakdown of Firefox full installer Mike Hommey schrieb: The only way to generate them is to run firefox. Well, there are other ways, but I don't think adding parts of the js engine and gecko to the installer is really a great idea. Could the installer run the Firefox binary in some mode that would just generate those files and then exit? It definitely could. Keep in mind though that the last several times it was checked download size doesn't appear to significantly affect install success and that there are other areas identified that do. Robert ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
Mike Hommey schrieb: The only way to generate them is to run firefox. Well, there are other ways, but I don't think adding parts of the js engine and gecko to the installer is really a great idea. Could the installer run the Firefox binary in some mode that would just generate those files and then exit? KaiRo ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
Mike Hommey schrieb: Note a significant amount of the omni.ja and browser/omni.ja data is used for jsloader/jssubloader data: 4744949 and 1560499 bytes from those files are that. These jsloader/jssubloader data are there for startup benefits on Firefox first run (if the data wasn't there, it would be generated on the first run and stored in the user profile). This was added a while ago, and we've been wondering if the js parser speed had been improved enough for those to now be useless for a while. It seems to me there's a lot to gain in download size from stripping those if we can confirm they are not helping in a significant way anymore. Who wants to check? Also, if we can't get decent startup without them, why do we need to download those pieces and not generate them with the installer in some fashion? KaiRo ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On Tue, Oct 14, 2014 at 7:09 PM, Mike Hommey m...@glandium.org wrote: I'm not saying we shouldn't strive for better, but I'm questioning the fact that download size would be affecting our growth. If the download size of our competitors is not affecting theirs, why would it affect ours? (and again, the premise is an interrogation) Who says it's not affecting their growth? Smaller download size is always going to be an advantage. Nothing magical happens at the point when you pass your competitors size. For example, for a user that already has Chrome but is curious to test Firefox, the download size of Chrome is effectively 0 bytes. And for a user that is determined to switch browser but is considering both Chrome and Firefox, if they try firefox first but they give up because the download is taking too long, at the point once they decide to try Chrome instead but realize that Chrome is going to take just as long to download, there's a risk that they'll stick with chrome because they are now already at the Chrome website. Or for a user that already has IE but would like to switch to Chrome or Firefox, but realize that both are too big downloads, might just give up and stick with IE. Any time that a user starts downloading Firefox, but gives up because the download is taking too long, is an opportunity where a smaller download could have helped. So I don't think we should focus too much on competitors download size. We should instead focus on our own funnel. / Jonas ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
Le 15/10/2014 04:09, Mike Hommey a écrit : On Tue, Oct 14, 2014 at 09:03:30PM -0400, Ehsan Akhgari wrote: Coming from a country with typically slow Internet connections, I strongly disagree. We should absolutely strive to be better than the competition by providing a smaller download size. Only matching the competition should be the minimum bar. :) I'm not saying we shouldn't strive for better, but I'm questioning the fact that download size would be affecting our growth. If the download size of our competitors is not affecting theirs, why would it affect ours? (and again, the premise is an interrogation) Mike Hi Maybe we should take these things into account too?: 1/ A lot of the people I know that have Chrome didn't download it, it came bundled with the computer/flash/antivirus/acrobat/google earth... Downloading is more crucial to us than it is to Google because it is our only distribution channel, that's why I think we need to do better than the competition there. 2/ Maybe Google has better infrastructure than us and can provide better download speed across the globe? I haven't noticed download speed difference between the Chrome and Firefox installers from Paris, but are we sure that out download speed is as efficient as it could be in Kenya, India or China? 3/ Our other big competitors are IE and Safari and they also come bundled with Windows, so no downloading needed either for them Regards, Pascal ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
- Original Message - From: Jonas Sicking jo...@sicking.cc To: Mike Hommey m...@glandium.org Cc: Chris More cm...@mozilla.com, Ehsan Akhgari ehsan.akhg...@gmail.com, dev-platform dev-platform@lists.mozilla.org, Daniel Veditz dved...@mozilla.com Sent: Wednesday, October 15, 2014 12:46:43 AM Subject: Re: Breakdown of Firefox full installer On Tue, Oct 14, 2014 at 7:09 PM, Mike Hommey m...@glandium.org wrote: I'm not saying we shouldn't strive for better, but I'm questioning the fact that download size would be affecting our growth. If the download size of our competitors is not affecting theirs, why would it affect ours? (and again, the premise is an interrogation) Who says it's not affecting their growth? I'm certain it is though I think it would be a good thing to question to what extent as well as where we should focus our resources. Except for the stub installer download we do know that it can't be more than 10% of installation attempts that have enough of an internet connection to provide data (e.g. if they can download the stub installer I suspect that they can send the data points back). I also know from analyzing the stub installer data that the vast majority of that 10% are cancelled installations and that if you break those down to 5 minute increments the vast majority are in the first 5 minutes. I believe the same is true if broken down in 1 minute increments though I'd prefer to verify this first. My system for analyzing the data is currently out of commission or I'd verify it now. Smaller download size is always going to be an advantage. Nothing magical happens at the point when you pass your competitors size. For example, for a user that already has Chrome but is curious to test Firefox, the download size of Chrome is effectively 0 bytes. And for a user that is determined to switch browser but is considering both Chrome and Firefox, if they try firefox first but they give up because the download is taking too long, at the point once they decide to try Chrome instead but realize that Chrome is going to take just as long to download, there's a risk that they'll stick with chrome because they are now already at the Chrome website. Or for a user that already has IE but would like to switch to Chrome or Firefox, but realize that both are too big downloads, might just give up and stick with IE. This is one of the reasons we moved to the stub installer so they can quickly perform the download quickly and then initiate the download / installation quickly. Any time that a user starts downloading Firefox, but gives up because the download is taking too long, is an opportunity where a smaller download could have helped. Not denying it but we do report this info in the stub installer and performed A / B testing where B was 3 MB larger and conversion rate was not significantly affected. So I don't think we should focus too much on competitors download size. We should instead focus on our own funnel. Agreed and as I said, We should always strive to do the best that we possibly can and focus our efforts on the areas with the greatest impact. Robert / Jonas ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On Tue, Oct 14, 2014 at 3:37 AM, Chris Hofmann chofm...@mozilla.com wrote: Now that we have a strategy for putting developer tools in there own release What's that strategy? It could be very bad for us if Dev Tools didn't just work for Web devs because they downloaded a non-developer build, when Dev Tools just work in Chrome and IE. On the topic of browser features implemented using HTML+JS: We should probably be more careful about importing JS libraries developed for Web use, because code paths for other browsers or legacy versions of Firefox are bloat in JS libs used inside Firefox. See, for example, Loop including a polyfill for TextDecoder/TextEncoder (which we have a built-in implementation of): https://bugzilla.mozilla.org/show_bug.cgi?id=1071536 -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On Wed, Oct 15, 2014 at 01:25:02PM +0300, Henri Sivonen wrote: On Tue, Oct 14, 2014 at 3:37 AM, Chris Hofmann chofm...@mozilla.com wrote: Now that we have a strategy for putting developer tools in there own release What's that strategy? It could be very bad for us if Dev Tools didn't just work for Web devs because they downloaded a non-developer build, when Dev Tools just work in Chrome and IE. On the topic of browser features implemented using HTML+JS: We should probably be more careful about importing JS libraries developed for Web use, because code paths for other browsers or legacy versions of Firefox are bloat in JS libs used inside Firefox. See, for example, Loop including a polyfill for TextDecoder/TextEncoder (which we have a built-in implementation of): https://bugzilla.mozilla.org/show_bug.cgi?id=1071536 Not mentioning that we probably have different versions of the same libs, and/or different implementations of the same kind of helpers. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
Robert Strong wrote: Another example, if the omni.jar is not compressed the installer can compress it about as well as if they were individual files and the minimal compression currently used by omni.jar makes it so the installer is not able to compress the omni.jar nearly as well which increases the installer size. As I recall we used to do this but it turns out that the size of the omni.jar itself is a large factor on startup performance - the time saved by reading less data from disk far outweighs the time spent decompressing it. -- Warning: May contain traces of nuts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
Gregory Szorc wrote: If you treat all files from those two archives as a single compression context Aha, this was the bit I was overlooking. Sorry for the confusion. -- Warning: May contain traces of nuts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
RE: Breakdown of Firefox full installer
Currently, there is no specific strategy except leaning what makes up the size of the installer and what is the driver of the increased size each release. After we learn that and people know what is causing the increased size, we should do some more A/B testing to see if a larger or smaller download size has an impact on conversations. Regardless of the magnitude of the change, anything we learn is positive. Even if nothing is ever changed, learning and understanding is a positive experience. If we ever collectively decided to do something different, we would have to evaluate the cost/benefit of doing so and again test it. So, for now, the strategy is learning. Whatever we learn will probably points us in the next direction. Thanks, Chris div Original message /divdivFrom: Henri Sivonen hsivo...@hsivonen.fi /divdivDate:10/15/2014 3:25 AM (GMT-08:00) /divdivTo: chofm...@mozilla.org /divdivCc: Kyle Huey m...@kylehuey.com,Chris More cm...@mozilla.com,dev-platform dev-platform@lists.mozilla.org /divdivSubject: Re: Breakdown of Firefox full installer /divdiv /divOn Tue, Oct 14, 2014 at 3:37 AM, Chris Hofmann chofm...@mozilla.com wrote: Now that we have a strategy for putting developer tools in there own release What's that strategy? It could be very bad for us if Dev Tools didn't just work for Web devs because they downloaded a non-developer build, when Dev Tools just work in Chrome and IE. On the topic of browser features implemented using HTML+JS: We should probably be more careful about importing JS libraries developed for Web use, because code paths for other browsers or legacy versions of Firefox are bloat in JS libs used inside Firefox. See, for example, Loop including a polyfill for TextDecoder/TextEncoder (which we have a built-in implementation of): https://bugzilla.mozilla.org/show_bug.cgi?id=1071536 -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 10/15/14 3:25 AM, Henri Sivonen wrote: On Tue, Oct 14, 2014 at 3:37 AM, Chris Hofmann chofm...@mozilla.com wrote: Now that we have a strategy for putting developer tools in there own release What's that strategy? It could be very bad for us if Dev Tools didn't just work for Web devs because they downloaded a non-developer build, when Dev Tools just work in Chrome and IE. On the topic of browser features implemented using HTML+JS: We should probably be more careful about importing JS libraries developed for Web use, because code paths for other browsers or legacy versions of Firefox are bloat in JS libs used inside Firefox. See, for example, Loop including a polyfill for TextDecoder/TextEncoder (which we have a built-in implementation of): https://bugzilla.mozilla.org/show_bug.cgi?id=1071536 Bug 977292 - On demand application components https://bugzilla.mozilla.org/show_bug.cgi?id=977292 Paradigm shift for Mozilla, but 10 out of 10 engineers agree: rm has a better compression ratio than any algorithm. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 10/14/2014 02:09 AM, Kyle Huey wrote:
The simplest way to break the installer down is by the files in
it.
e.g. http://khuey.pastebin.mozilla.org/6781501
For future reference:
mozilla@KHUEY-19294 /c/dev/scratch $ wget
ftp://ftp.mozilla.org/pub/firefox/releases/latest/win32/en-US/Firefox%20Setup%2032.0.3.exe
- --16:59:06--
ftp://ftp.mozilla.org/pub/firefox/releases/latest/win32/en-US/Firefox%20Setup%2032.0.3.exe
= `Firefox Setup 32.0.3.exe' Resolving ftp.mozilla.org...
63.245.215.46, 63.245.215.56 Connecting to
ftp.mozilla.org|63.245.215.46|:21... connected. Logging in as
anonymous ... Logged in! == SYST ... done.== PWD ... done.
== TYPE I ... done. == CWD
/pub/firefox/releases/latest/win32/en-US ... done.
== PASV ... done.== RETR Firefox Setup 32.0.3.exe ... done.
Length: 35,285,328 (34M) (unauthoritative)
100%[] 35,285,328 967.16K/s
ETA 00:00
17:00:07 (617.58 KB/s) - `Firefox Setup 32.0.3.exe' saved
[35285328]
mozilla@KHUEY-19294 /c/dev/scratch $ ls Firefox Setup 32.0.3.exe
mozilla@KHUEY-19294 /c/dev/scratch $ 7z x Firefox\ Setup\
32.0.3.exe
7-Zip 4.42 Copyright (c) 1999-2006 Igor Pavlov 2006-05-14
Processing archive: Firefox Setup 32.0.3.exe
Extracting core\precomplete Extracting core\removed-files
Extracting
core\browser\extensions\{972ce4c6-7e08-4474-a285-3208198ce6fd}\icon.
png
Extracting core\browser\chrome.manifest Extracting
core\browser\components\components.manifest Extracting
core\browser\searchplugins\amazondotcom.xml Extracting
core\browser\searchplugins\bing.xml Extracting
core\browser\blocklist.xml Extracting
core\browser\searchplugins\eBay.xml Extracting
core\browser\searchplugins\google.xml Extracting
core\browser\searchplugins\twitter.xml Extracting
core\browser\searchplugins\wikipedia.xml Extracting
core\browser\searchplugins\yahoo.xml Extracting
core\defaults\pref\channel-prefs.js Extracting
core\application.ini Extracting
core\browser\crashreporter-override.ini Extracting
core\crashreporter.ini Extracting core\platform.ini Extracting
core\update-settings.ini Extracting core\updater.ini Extracting
core\webapprt\webapprt.ini Extracting core\crashreporter.exe
Extracting core\firefox.exe Extracting core\uninstall\helper.exe
Extracting core\maintenanceservice.exe Extracting
core\maintenanceservice_installer.exe Extracting
core\plugin-container.exe Extracting core\plugin-hang-ui.exe
Extracting setup.exe Extracting core\updater.exe Extracting
core\webapp-uninstaller.exe Extracting core\webapprt-stub.exe
Extracting core\AccessibleMarshal.dll Extracting
core\breakpadinjector.dll Extracting
core\browser\components\browsercomps.dll Extracting
core\D3DCompiler_43.dll Extracting core\d3dcompiler_46.dll
Extracting core\freebl3.dll Extracting core\gkmedias.dll
Extracting core\icudt52.dll Extracting core\icuin52.dll
Extracting core\icuuc52.dll Extracting core\libEGL.dll Extracting
core\libGLESv2.dll Extracting core\mozalloc.dll Extracting
core\mozglue.dll Extracting core\mozjs.dll Extracting
core\msvcp100.dll Extracting core\msvcr100.dll Extracting
core\nss3.dll Extracting core\nssckbi.dll Extracting
core\nssdbm3.dll Extracting core\softokn3.dll Extracting
core\xul.dll Extracting core\dictionaries\en-US.aff Extracting
core\freebl3.chk Extracting core\nssdbm3.chk Extracting
core\softokn3.chk Extracting core\dictionaries\en-US.dic
Extracting core\browser\omni.ja Extracting core\webapprt\omni.ja
Extracting core\omni.ja Extracting core\dependentlibs.list
Extracting
core\browser\extensions\{972ce4c6-7e08-4474-a285-3208198ce6fd}\install.rdf
Extracting core\webapprt
Extracting core\uninstall Extracting core\dictionaries Extracting
core\defaults\pref Extracting core\defaults Extracting
core\browser\searchplugins Extracting
core\browser\extensions\{972ce4c6-7e08-4474-a285-3208198ce6fd}
Extracting core\browser\extensions Extracting
core\browser\components Extracting core\browser Extracting core
Everything is Ok
mozilla@KHUEY-19294 /c/dev/scratch $ rm Firefox\ Setup\ 32.0.3.exe
mozilla@KHUEY-19294 /c/dev/scratch $ find . -type f -printf %s@
%p\n | sort -nr 25027184@ ./core/xul.dll 11174488@ ./core/omni.ja
10397296@ ./core/icudt52.dll 8930647@ ./core/browser/omni.ja
4877424@ ./core/gkmedias.dll 3715184@ ./core/mozjs.dll 3231696@
./core/d3dcompiler_46.dll 2106216@ ./core/D3DCompiler_43.dll
1803376@ ./core/nss3.dll 1023600@ ./core/icuin52.dll 897688@
./core/uninstall/helper.exe 800368@ ./core/icuuc52.dll 770384@
./core/msvcr100.dll 684984@ ./setup.exe 638064@
./core/libGLESv2.dll 622592@ ./core/dictionaries/en-US.dic 421200@
./core/msvcp100.dll 413296@ ./core/nssckbi.dll 331376@
./core/freebl3.dll 275568@ ./core/firefox.exe 273008@
./core/updater.exe 198232@ ./core/maintenanceservice_installer.exe
150128@ ./core/softokn3.dll 140400@
Re: Breakdown of Firefox full installer
On 2014-10-13 8:48 PM, Mike Hommey wrote: Note a significant amount of the omni.ja and browser/omni.ja data is used for jsloader/jssubloader data: 4744949 and 1560499 bytes from those files are that. These jsloader/jssubloader data are there for startup benefits on Firefox first run (if the data wasn't there, it would be generated on the first run and stored in the user profile). This was added a while ago, and we've been wondering if the js parser speed had been improved enough for those to now be useless for a while. It seems to me there's a lot to gain in download size from stripping those if we can confirm they are not helping in a significant way anymore. Who wants to check? Aaron Klotz (aklotz) has done some preliminary work on this in bug 873638. I'm planning to take this on in this quarter, unless other priorities intervene. - irving - ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 10/13/2014 9:25 PM, Chris Peterson wrote: Going forward, it would be interesting to see a dashboard track Firefox installer size every day (or show every changeset's delta on Treeherder). We used to have http://arewesmallyet.com -- I found references to it as late as a year ago but it seems to be someone else now. I think this is the code for it: https://github.com/ttaubert/arewesmallyet. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 10/13/2014 4:54 PM, Chris More wrote: For example, the win32 installer for Firefox 32 is 34MB. Remember the days when Asa would jump all over people for breaking the 5Mb barrier? https://wiki.mozilla.org/Download_Size -Dan Veditz ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
Very interesting. When Firefox 4 was launched, it was 12MB. When Australis was launched it was 28MB. Now, Firefox 33 is 35MB. That's almost a 200% increase. I did an A/B test last year when the installer was 22MB and there was a strong correlation between average internet speed in a specific region of the world and install-rate of Firefox. If world-wide internet speeds are not increasing faster than the growth of the installer, it could be having a negative impact on adoption of the product. By how much? No one is for sure as the A/B test was just testing the current size of Firefox and one 3MB bigger. The conclusion of the test was that 22mB vs 25MB didn't have a statistical difference in conversion rate, but now a year later, we are more than 10MB bigger. It looks like everyone provided helpful information and this is a great start. I am going to work on creating a document to help to quantify what are the drivers in the growth of the installer. After that we can decide what does this tell us and if the growth of the installer has negative impacts. Thanks everyone! Chris - Original Message - From: Daniel Veditz dved...@mozilla.com To: Chris More cm...@mozilla.com, dev-platform@lists.mozilla.org Sent: Tuesday, October 14, 2014 9:27:52 AM Subject: Re: Breakdown of Firefox full installer On 10/13/2014 4:54 PM, Chris More wrote: For example, the win32 installer for Firefox 32 is 34MB. Remember the days when Asa would jump all over people for breaking the 5Mb barrier? https://wiki.mozilla.org/Download_Size -Dan Veditz ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 10/14/14 2:20 AM, Robert Strong wrote: * (Countries' average) Internet speed dramatically affected conversion rates: 70% success in the fastest countries and 30% in the slowest countries. Note that these conversion rates are from download to hitting the first run page and there are several factors that dramatically lower the conversion rate outside of the actual stub installer portion of the process. When just measuring the conversion rate from clicking install in the stub installer the conversion rate is actually close to 90%. Interesting; this is what I was asking about in my other post. If we're getting ~90% conversion in the stub installer, that implies download size improvements may have little effect. A max of the missing 10%, and presumably some portion of that is due to other factors. Justin ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On Tue, Oct 14, 2014 at 11:11:01AM -0700, Chris More wrote: Very interesting. When Firefox 4 was launched, it was 12MB. When Australis was launched it was 28MB. Now, Firefox 33 is 35MB. That's almost a 200% increase. I did an A/B test last year when the installer was 22MB and there was a strong correlation between average internet speed in a specific region of the world and install-rate of Firefox. If world-wide internet speeds are not increasing faster than the growth of the installer, it could be having a negative impact on adoption of the product. By how much? No one is for sure as the A/B test was just testing the current size of Firefox and one 3MB bigger. The conclusion of the test was that 22mB vs 25MB didn't have a statistical difference in conversion rate, but now a year later, we are more than 10MB bigger. It looks like everyone provided helpful information and this is a great start. I am going to work on creating a document to help to quantify what are the drivers in the growth of the installer. After that we can decide what does this tell us and if the growth of the installer has negative impacts. On the other hand, what is the download size of the downloadable alternatives to Firefox? What is their adoption in regions with poor internet speeds? If the answer to both those questions is bigger, (which I genuinely don't know if it is) there is nothing wrong with our download size. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
Great question and we've discussed the same thing last year. Last year, Chrome was about the same size if not slightly bigger and that looks to be the same case now. It is still worthwhile to understand about our installer size, the driver of growth each release, and if there is any impact. Chrome full installer is currently 40MB on Windows: https://dl.google.com/update2/installers/ChromeStandaloneSetup.exe It is not as easy to look at their growth because I cannot find any FTP directories or other websites that have their installer sizes over time. Chris - Original Message - From: Mike Hommey m...@glandium.org To: Chris More cm...@mozilla.com Cc: Daniel Veditz dved...@mozilla.com, dev-platform@lists.mozilla.org Sent: Tuesday, October 14, 2014 3:53:34 PM Subject: Re: Breakdown of Firefox full installer On Tue, Oct 14, 2014 at 11:11:01AM -0700, Chris More wrote: Very interesting. When Firefox 4 was launched, it was 12MB. When Australis was launched it was 28MB. Now, Firefox 33 is 35MB. That's almost a 200% increase. I did an A/B test last year when the installer was 22MB and there was a strong correlation between average internet speed in a specific region of the world and install-rate of Firefox. If world-wide internet speeds are not increasing faster than the growth of the installer, it could be having a negative impact on adoption of the product. By how much? No one is for sure as the A/B test was just testing the current size of Firefox and one 3MB bigger. The conclusion of the test was that 22mB vs 25MB didn't have a statistical difference in conversion rate, but now a year later, we are more than 10MB bigger. It looks like everyone provided helpful information and this is a great start. I am going to work on creating a document to help to quantify what are the drivers in the growth of the installer. After that we can decide what does this tell us and if the growth of the installer has negative impacts. On the other hand, what is the download size of the downloadable alternatives to Firefox? What is their adoption in regions with poor internet speeds? If the answer to both those questions is bigger, (which I genuinely don't know if it is) there is nothing wrong with our download size. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
Also, it may be challenging to contact the Chrome team and ask them for their conversion rates to install by country, but I would imagine they are probably facing similar challenges. They may also have other methods of distribution to get around the constraints or have enough marketing $$ to offsets organic growth in emerging markets. If anyone does have any non-Firefox information to share, it would be helpful. This is probably a challenge with any kind of download, browser or not. Chris - Original Message - From: Mike Hommey m...@glandium.org To: Chris More cm...@mozilla.com Cc: Daniel Veditz dved...@mozilla.com, dev-platform@lists.mozilla.org Sent: Tuesday, October 14, 2014 3:53:34 PM Subject: Re: Breakdown of Firefox full installer On Tue, Oct 14, 2014 at 11:11:01AM -0700, Chris More wrote: Very interesting. When Firefox 4 was launched, it was 12MB. When Australis was launched it was 28MB. Now, Firefox 33 is 35MB. That's almost a 200% increase. I did an A/B test last year when the installer was 22MB and there was a strong correlation between average internet speed in a specific region of the world and install-rate of Firefox. If world-wide internet speeds are not increasing faster than the growth of the installer, it could be having a negative impact on adoption of the product. By how much? No one is for sure as the A/B test was just testing the current size of Firefox and one 3MB bigger. The conclusion of the test was that 22mB vs 25MB didn't have a statistical difference in conversion rate, but now a year later, we are more than 10MB bigger. It looks like everyone provided helpful information and this is a great start. I am going to work on creating a document to help to quantify what are the drivers in the growth of the installer. After that we can decide what does this tell us and if the growth of the installer has negative impacts. On the other hand, what is the download size of the downloadable alternatives to Firefox? What is their adoption in regions with poor internet speeds? If the answer to both those questions is bigger, (which I genuinely don't know if it is) there is nothing wrong with our download size. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
Gregory Szorc wrote: If you are looking for ideas on how to reduce download size, the way omni.ja is included in the installer could be reduced by 4+ MB. Both omni.ja and browser/omni.ja are zip archives, where each file has a separate compression context. If you treat all files from those two archives as a single compression context (like tar+bz2) and stuff them in a single archive, you get size reductions of 4+ MB due to the compression engine sharing state between files. We can't install omni.ja like this on client systems because it is bad for performance (we want the ability to extract individual files without reading a large compression context - this is a benefit of the zip format). But we could ship the files optimized for size (or have installer's compression handle the files individually) and have the installer re-encode them to omni.ja so they are optimized for performance. I'm not sure what you're trying to say here. As far as I can see what you're suggesting is using some RAR-like rather than ZIP-like format for the installer. Why would the way you're compressing the two omni.ja files into the installer affect the installed omni.ja files? -- Warning: May contain traces of nuts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
I found my previous analysis of the stub installer: Stub installs without a Firefox profile and not installing on top of an existing install (e.g. new installs) have a 90.32% success rate for Firefox 30 during the first 2 weeks starting the Friday after release. Robert - Original Message - From: Justin Dolske dol...@mozilla.com To: dev-platform@lists.mozilla.org Sent: Tuesday, October 14, 2014 12:34:53 PM Subject: Re: Breakdown of Firefox full installer On 10/14/14 2:20 AM, Robert Strong wrote: * (Countries' average) Internet speed dramatically affected conversion rates: 70% success in the fastest countries and 30% in the slowest countries. Note that these conversion rates are from download to hitting the first run page and there are several factors that dramatically lower the conversion rate outside of the actual stub installer portion of the process. When just measuring the conversion rate from clicking install in the stub installer the conversion rate is actually close to 90%. Interesting; this is what I was asking about in my other post. If we're getting ~90% conversion in the stub installer, that implies download size improvements may have little effect. A max of the missing 10%, and presumably some portion of that is due to other factors. Justin ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
Another example, if the omni.jar is not compressed the installer can compress it about as well as if they were individual files and the minimal compression currently used by omni.jar makes it so the installer is not able to compress the omni.jar nearly as well which increases the installer size. Robert - Original Message - From: Gregory Szorc g...@mozilla.com To: Neil n...@parkwaycc.co.uk, dev-platform@lists.mozilla.org Sent: Tuesday, October 14, 2014 5:31:54 PM Subject: Re: Breakdown of Firefox full installer On 10/14/14 5:12 PM, Neil wrote: Gregory Szorc wrote: If you are looking for ideas on how to reduce download size, the way omni.ja is included in the installer could be reduced by 4+ MB. Both omni.ja and browser/omni.ja are zip archives, where each file has a separate compression context. If you treat all files from those two archives as a single compression context (like tar+bz2) and stuff them in a single archive, you get size reductions of 4+ MB due to the compression engine sharing state between files. We can't install omni.ja like this on client systems because it is bad for performance (we want the ability to extract individual files without reading a large compression context - this is a benefit of the zip format). But we could ship the files optimized for size (or have installer's compression handle the files individually) and have the installer re-encode them to omni.ja so they are optimized for performance. I'm not sure what you're trying to say here. As far as I can see what you're suggesting is using some RAR-like rather than ZIP-like format for the installer. Why would the way you're compressing the two omni.ja files into the installer affect the installed omni.ja files? The more data you throw into the compression engine, the greater the chances there will be redundant data and you won't have duplicate representations of that data inside the compressed result. There's a lot of data inside both omni.ja and it turns out that many forms of compression will find those duplicate patterns and prevent the redundancy. Also, we're not talking about changing omni.ja as it exists in the installed application: only how data within omni.ja is packaged and shipped in the installer. Discussions on whether omni.ja should be e.g. lzma or laid out differently on disk are orthogonal but still important to have (because they impact performance, especially start times). ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 2014-10-14, 7:25 PM, Chris More wrote: Great question and we've discussed the same thing last year. Last year, Chrome was about the same size if not slightly bigger and that looks to be the same case now. It is still worthwhile to understand about our installer size, the driver of growth each release, and if there is any impact. Chrome full installer is currently 40MB on Windows: https://dl.google.com/update2/installers/ChromeStandaloneSetup.exe It is not as easy to look at their growth because I cannot find any FTP directories or other websites that have their installer sizes over time. You can find them here for Chromium: http://commondatastorage.googleapis.com/chromium-browser-continuous/index.html ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 2014-10-14, 6:53 PM, Mike Hommey wrote: On Tue, Oct 14, 2014 at 11:11:01AM -0700, Chris More wrote: Very interesting. When Firefox 4 was launched, it was 12MB. When Australis was launched it was 28MB. Now, Firefox 33 is 35MB. That's almost a 200% increase. I did an A/B test last year when the installer was 22MB and there was a strong correlation between average internet speed in a specific region of the world and install-rate of Firefox. If world-wide internet speeds are not increasing faster than the growth of the installer, it could be having a negative impact on adoption of the product. By how much? No one is for sure as the A/B test was just testing the current size of Firefox and one 3MB bigger. The conclusion of the test was that 22mB vs 25MB didn't have a statistical difference in conversion rate, but now a year later, we are more than 10MB bigger. It looks like everyone provided helpful information and this is a great start. I am going to work on creating a document to help to quantify what are the drivers in the growth of the installer. After that we can decide what does this tell us and if the growth of the installer has negative impacts. On the other hand, what is the download size of the downloadable alternatives to Firefox? What is their adoption in regions with poor internet speeds? If the answer to both those questions is bigger, (which I genuinely don't know if it is) there is nothing wrong with our download size. Coming from a country with typically slow Internet connections, I strongly disagree. We should absolutely strive to be better than the competition by providing a smaller download size. Only matching the competition should be the minimum bar. :) ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On Tue, Oct 14, 2014 at 09:03:30PM -0400, Ehsan Akhgari wrote: On 2014-10-14, 6:53 PM, Mike Hommey wrote: On Tue, Oct 14, 2014 at 11:11:01AM -0700, Chris More wrote: Very interesting. When Firefox 4 was launched, it was 12MB. When Australis was launched it was 28MB. Now, Firefox 33 is 35MB. That's almost a 200% increase. I did an A/B test last year when the installer was 22MB and there was a strong correlation between average internet speed in a specific region of the world and install-rate of Firefox. If world-wide internet speeds are not increasing faster than the growth of the installer, it could be having a negative impact on adoption of the product. By how much? No one is for sure as the A/B test was just testing the current size of Firefox and one 3MB bigger. The conclusion of the test was that 22mB vs 25MB didn't have a statistical difference in conversion rate, but now a year later, we are more than 10MB bigger. It looks like everyone provided helpful information and this is a great start. I am going to work on creating a document to help to quantify what are the drivers in the growth of the installer. After that we can decide what does this tell us and if the growth of the installer has negative impacts. On the other hand, what is the download size of the downloadable alternatives to Firefox? What is their adoption in regions with poor internet speeds? If the answer to both those questions is bigger, (which I genuinely don't know if it is) there is nothing wrong with our download size. Coming from a country with typically slow Internet connections, I strongly disagree. We should absolutely strive to be better than the competition by providing a smaller download size. Only matching the competition should be the minimum bar. :) I'm not saying we shouldn't strive for better, but I'm questioning the fact that download size would be affecting our growth. If the download size of our competitors is not affecting theirs, why would it affect ours? (and again, the premise is an interrogation) Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 2014-10-14, 10:09 PM, Mike Hommey wrote: On Tue, Oct 14, 2014 at 09:03:30PM -0400, Ehsan Akhgari wrote: On 2014-10-14, 6:53 PM, Mike Hommey wrote: On Tue, Oct 14, 2014 at 11:11:01AM -0700, Chris More wrote: Very interesting. When Firefox 4 was launched, it was 12MB. When Australis was launched it was 28MB. Now, Firefox 33 is 35MB. That's almost a 200% increase. I did an A/B test last year when the installer was 22MB and there was a strong correlation between average internet speed in a specific region of the world and install-rate of Firefox. If world-wide internet speeds are not increasing faster than the growth of the installer, it could be having a negative impact on adoption of the product. By how much? No one is for sure as the A/B test was just testing the current size of Firefox and one 3MB bigger. The conclusion of the test was that 22mB vs 25MB didn't have a statistical difference in conversion rate, but now a year later, we are more than 10MB bigger. It looks like everyone provided helpful information and this is a great start. I am going to work on creating a document to help to quantify what are the drivers in the growth of the installer. After that we can decide what does this tell us and if the growth of the installer has negative impacts. On the other hand, what is the download size of the downloadable alternatives to Firefox? What is their adoption in regions with poor internet speeds? If the answer to both those questions is bigger, (which I genuinely don't know if it is) there is nothing wrong with our download size. Coming from a country with typically slow Internet connections, I strongly disagree. We should absolutely strive to be better than the competition by providing a smaller download size. Only matching the competition should be the minimum bar. :) I'm not saying we shouldn't strive for better, but I'm questioning the fact that download size would be affecting our growth. If the download size of our competitors is not affecting theirs, why would it affect ours? (and again, the premise is an interrogation) That's a fair objection. One possible explanation for that would be the hypothesis that they have more directly focused on markets where broadband Internet is prevalent through, let's say TV ads in the case of Chrome. Of course I don't know whether that's true, but I don't think we can necessarily assume a similar growth impact potential here for different browsers. Cheers, Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Breakdown of Firefox full installer
- Original Message - From: Mike Hommey m...@glandium.org To: Ehsan Akhgari ehsan.akhg...@gmail.com Cc: Chris More cm...@mozilla.com, dev-platform@lists.mozilla.org, Daniel Veditz dved...@mozilla.com Sent: Tuesday, October 14, 2014 7:09:30 PM Subject: Re: Breakdown of Firefox full installer On Tue, Oct 14, 2014 at 09:03:30PM -0400, Ehsan Akhgari wrote: On 2014-10-14, 6:53 PM, Mike Hommey wrote: On Tue, Oct 14, 2014 at 11:11:01AM -0700, Chris More wrote: Very interesting. When Firefox 4 was launched, it was 12MB. When Australis was launched it was 28MB. Now, Firefox 33 is 35MB. That's almost a 200% increase. I did an A/B test last year when the installer was 22MB and there was a strong correlation between average internet speed in a specific region of the world and install-rate of Firefox. If world-wide internet speeds are not increasing faster than the growth of the installer, it could be having a negative impact on adoption of the product. By how much? No one is for sure as the A/B test was just testing the current size of Firefox and one 3MB bigger. The conclusion of the test was that 22mB vs 25MB didn't have a statistical difference in conversion rate, but now a year later, we are more than 10MB bigger. It looks like everyone provided helpful information and this is a great start. I am going to work on creating a document to help to quantify what are the drivers in the growth of the installer. After that we can decide what does this tell us and if the growth of the installer has negative impacts. On the other hand, what is the download size of the downloadable alternatives to Firefox? What is their adoption in regions with poor internet speeds? If the answer to both those questions is bigger, (which I genuinely don't know if it is) there is nothing wrong with our download size. Coming from a country with typically slow Internet connections, I strongly disagree. We should absolutely strive to be better than the competition by providing a smaller download size. Only matching the competition should be the minimum bar. :) I'm not saying we shouldn't strive for better, but I'm questioning the fact that download size would be affecting our growth. If the download size of our competitors is not affecting theirs, why would it affect ours? (and again, the premise is an interrogation) Also agreed. The study with the 3 MB increase shows that it didn't significantly affect conversions. I personally think it would be better to allocate resources to make the stub installer download phase more resilient for the poor network connectivity case though resourcing this work is not possible from my end with the other work currently going on. It would also be a good thing to fully understand and show why there is such a dramatic difference between the entire conversion process (~70%) vs. the stub installer process (~90%). As I understand it a large portion is due to bots repeatedly downloading the full installer without ever performing an install and the first run page not being shown under certain conditions though I wouldn't be surprised at all if there were several others. I've repeatedly asked for this and asked that when that number is presented to people that caveat is included since it is consistently assumed that if we just changed the stub installer we could improve the install process conversion rate by an amount that isn't even available to the stub installer. Robert ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 2014-10-14 10:14 PM, Ehsan Akhgari wrote: On 2014-10-14, 10:09 PM, Mike Hommey wrote: I'm not saying we shouldn't strive for better, but I'm questioning the fact that download size would be affecting our growth. If the download size of our competitors is not affecting theirs, why would it affect ours? (and again, the premise is an interrogation) That's a fair objection. One possible explanation for that would be the hypothesis that they have more directly focused on markets where broadband Internet is prevalent through, let's say TV ads in the case of Chrome. Of course I don't know whether that's true, but I don't think we can necessarily assume a similar growth impact potential here for different browsers. For whatever it's worth I think we're conflating bits-on-the-disk with bits-on-the-wire here, which isn't necessarily the how things need to work. - mhoye ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 10/13/14 4:54 PM, Chris More wrote: Does anyone know or could any of you create a breakdown of the major blocks of the Firefox installer and each of their respective sizes or percentage of the whole? For example, the win32 installer for Firefox 32 is 34MB. The Firefox Growth team [1] like to know of that 34MB, what is the percentage or size of each of the components within the 34MB. As for the granularity of the breakdown, it would be by some logic way of breaking down Firefox. For example, SpiderMonkey, tools, XUL, etc. I'll leave the granularity up to you on what you consider a logic block to quantify together. Why am I asking this? The win32 Firefox full installer continues to grow (see attachment) each release and it has been on an increasing growth since Firefox 29. Like anything on the web, the time it takes to download something (webpage, binary file, etc.) affects the key conversion rate by some amount. The Firefox Growth team has a project to understand what features/changes in Firefox are contributing to the growth or size of the installer. We've asked a few times previously, but it doesn't look like the documentation or analysis exist. Would anyone be able to take on this project as it would be very helpful to the team? I am imagining a pie chart of the the current installer and then a table of the name of each component, their size (KB or MB), and any additional meta data. If you are looking for ideas on how to reduce download size, the way omni.ja is included in the installer could be reduced by 4+ MB. Both omni.ja and browser/omni.ja are zip archives, where each file has a separate compression context. If you treat all files from those two archives as a single compression context (like tar+bz2) and stuff them in a single archive, you get size reductions of 4+ MB due to the compression engine sharing state between files. We can't install omni.ja like this on client systems because it is bad for performance (we want the ability to extract individual files without reading a large compression context - this is a benefit of the zip format). But we could ship the files optimized for size (or have installer's compression handle the files individually) and have the installer re-encode them to omni.ja so they are optimized for performance. I'm not sure if this has been considered or attempted before. Things are slightly complicated by the fact that omni.ja is a slightly customized zip format. We'd need to ship code to encode omni.ja. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On Mon, Oct 13, 2014 at 5:37 PM, Chris Hofmann chofm...@mozilla.com wrote: one thing to add on Kyle's analysis and numbers is to zip all the files back up individually, then measure the size of each, since text files are likely to have a higher compression rate than binary. Pretty much everything in there is binary. The largest text file is the en-US dictionary which is only 600K. - Kyle ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On Mon, Oct 13, 2014 at 05:09:41PM -0700, Kyle Huey wrote: On Mon, Oct 13, 2014 at 4:54 PM, Chris More cm...@mozilla.com wrote: Does anyone know or could any of you create a breakdown of the major blocks of the Firefox installer and each of their respective sizes or percentage of the whole? For example, the win32 installer for Firefox 32 is 34MB. The Firefox Growth team [1] like to know of that 34MB, what is the percentage or size of each of the components within the 34MB. As for the granularity of the breakdown, it would be by some logic way of breaking down Firefox. For example, SpiderMonkey, tools, XUL, etc. I'll leave the granularity up to you on what you consider a logic block to quantify together. Why am I asking this? The win32 Firefox full installer continues to grow (see attachment) each release and it has been on an increasing growth since Firefox 29. Like anything on the web, the time it takes to download something (webpage, binary file, etc.) affects the key conversion rate by some amount. The Firefox Growth team has a project to understand what features/changes in Firefox are contributing to the growth or size of the installer. We've asked a few times previously, but it doesn't look like the documentation or analysis exist. Would anyone be able to take on this project as it would be very helpful to the team? I am imagining a pie chart of the the current installer and then a table of the name of each component, their size (KB or MB), and any additional meta data. Thanks, Chris [1] https://wiki.mozilla.org/Growth_Team ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform The simplest way to break the installer down is by the files in it. e.g. http://khuey.pastebin.mozilla.org/6781501 File boundaries are a reasonably logical division to start with. Once you've identified which files are growing/being added over time we can explain what those are and where the growth comes from (in terms of features/etc). Note a significant amount of the omni.ja and browser/omni.ja data is used for jsloader/jssubloader data: 4744949 and 1560499 bytes from those files are that. These jsloader/jssubloader data are there for startup benefits on Firefox first run (if the data wasn't there, it would be generated on the first run and stored in the user profile). This was added a while ago, and we've been wondering if the js parser speed had been improved enough for those to now be useless for a while. It seems to me there's a lot to gain in download size from stripping those if we can confirm they are not helping in a significant way anymore. Who wants to check? Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 10/13/14 5:42 PM, Andreas Gal wrote: I looked at lzma2 a while ago for FFOS. I got pretty consistently 30% smaller omni.ja with that. We could add it pretty easily to our decompression code but it has slightly different memory behavior. This was discussed in the Including Adobe CMaps thread (https://groups.google.com/d/msg/mozilla.dev.platform/SOInehcZa2g/ezcQRH7k_moJ). tl;dr is we need per-file compression on client for performance reasons: whole archive compression for actual Gecko load is a non-starter. I see now that glandium mentioned adding functionality to the installer in that thread. I guess it was never acted upon. On Oct 13, 2014, at 5:39 PM, Gregory Szorc g...@mozilla.com wrote: On 10/13/14 4:54 PM, Chris More wrote: Does anyone know or could any of you create a breakdown of the major blocks of the Firefox installer and each of their respective sizes or percentage of the whole? For example, the win32 installer for Firefox 32 is 34MB. The Firefox Growth team [1] like to know of that 34MB, what is the percentage or size of each of the components within the 34MB. As for the granularity of the breakdown, it would be by some logic way of breaking down Firefox. For example, SpiderMonkey, tools, XUL, etc. I'll leave the granularity up to you on what you consider a logic block to quantify together. Why am I asking this? The win32 Firefox full installer continues to grow (see attachment) each release and it has been on an increasing growth since Firefox 29. Like anything on the web, the time it takes to download something (webpage, binary file, etc.) affects the key conversion rate by some amount. The Firefox Growth team has a project to understand what features/changes in Firefox are contributing to the growth or size of the installer. We've asked a few times previously, but it doesn't look like the documentation or analysis exist. Would anyone be able to take on this project as it would be very helpful to the team? I am imagining a pie chart of the the current installer and then a table of the name of each component, their size (KB or MB), and any additional meta data. If you are looking for ideas on how to reduce download size, the way omni.ja is included in the installer could be reduced by 4+ MB. Both omni.ja and browser/omni.ja are zip archives, where each file has a separate compression context. If you treat all files from those two archives as a single compression context (like tar+bz2) and stuff them in a single archive, you get size reductions of 4+ MB due to the compression engine sharing state between files. We can't install omni.ja like this on client systems because it is bad for performance (we want the ability to extract individual files without reading a large compression context - this is a benefit of the zip format). But we could ship the files optimized for size (or have installer's compression handle the files individually) and have the installer re-encode them to omni.ja so they are optimized for performance. I'm not sure if this has been considered or attempted before. Things are slightly complicated by the fact that omni.ja is a slightly customized zip format. We'd need to ship code to encode omni.ja. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On Mon, Oct 13, 2014 at 05:39:31PM -0700, Gregory Szorc wrote: On 10/13/14 4:54 PM, Chris More wrote: Does anyone know or could any of you create a breakdown of the major blocks of the Firefox installer and each of their respective sizes or percentage of the whole? For example, the win32 installer for Firefox 32 is 34MB. The Firefox Growth team [1] like to know of that 34MB, what is the percentage or size of each of the components within the 34MB. As for the granularity of the breakdown, it would be by some logic way of breaking down Firefox. For example, SpiderMonkey, tools, XUL, etc. I'll leave the granularity up to you on what you consider a logic block to quantify together. Why am I asking this? The win32 Firefox full installer continues to grow (see attachment) each release and it has been on an increasing growth since Firefox 29. Like anything on the web, the time it takes to download something (webpage, binary file, etc.) affects the key conversion rate by some amount. The Firefox Growth team has a project to understand what features/changes in Firefox are contributing to the growth or size of the installer. We've asked a few times previously, but it doesn't look like the documentation or analysis exist. Would anyone be able to take on this project as it would be very helpful to the team? I am imagining a pie chart of the the current installer and then a table of the name of each component, their size (KB or MB), and any additional meta data. If you are looking for ideas on how to reduce download size, the way omni.ja is included in the installer could be reduced by 4+ MB. Both omni.ja and browser/omni.ja are zip archives, where each file has a separate compression context. If you treat all files from those two archives as a single compression context (like tar+bz2) and stuff them in a single archive, you get size reductions of 4+ MB due to the compression engine sharing state between files. We can't install omni.ja like this on client systems because it is bad for performance (we want the ability to extract individual files without reading a large compression context - this is a benefit of the zip format). But we could ship the files optimized for size (or have installer's compression handle the files individually) and have the installer re-encode them to omni.ja so they are optimized for performance. I'm not sure if this has been considered or attempted before. Things are slightly complicated by the fact that omni.ja is a slightly customized zip format. We'd need to ship code to encode omni.ja. IIRC there is a bug about that. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On Mon, Oct 13, 2014 at 05:37:23PM -0700, Chris Hofmann wrote: On 10/13/14 5:09 PM, Kyle Huey wrote: On Mon, Oct 13, 2014 at 4:54 PM, Chris More cm...@mozilla.com wrote: Does anyone know or could any of you create a breakdown of the major blocks of the Firefox installer and each of their respective sizes or percentage of the whole? For example, the win32 installer for Firefox 32 is 34MB. The Firefox Growth team [1] like to know of that 34MB, what is the percentage or size of each of the components within the 34MB. As for the granularity of the breakdown, it would be by some logic way of breaking down Firefox. For example, SpiderMonkey, tools, XUL, etc. I'll leave the granularity up to you on what you consider a logic block to quantify together. Why am I asking this? The win32 Firefox full installer continues to grow (see attachment) each release and it has been on an increasing growth since Firefox 29. Like anything on the web, the time it takes to download something (webpage, binary file, etc.) affects the key conversion rate by some amount. The Firefox Growth team has a project to understand what features/changes in Firefox are contributing to the growth or size of the installer. We've asked a few times previously, but it doesn't look like the documentation or analysis exist. Would anyone be able to take on this project as it would be very helpful to the team? I am imagining a pie chart of the the current installer and then a table of the name of each component, their size (KB or MB), and any additional meta data. Thanks, Chris [1] https://wiki.mozilla.org/Growth_Team ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform The simplest way to break the installer down is by the files in it. e.g. http://khuey.pastebin.mozilla.org/6781501 File boundaries are a reasonably logical division to start with. Once you've identified which files are growing/being added over time we can explain what those are and where the growth comes from (in terms of features/etc). - Kyle one thing to add on Kyle's analysis and numbers is to zip all the files back up individually 7zip, not zip. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 10/13/14 4:54 PM, Chris More wrote: I am imagining a pie chart of the the current installer and then a table of the name of each component, their size (KB or MB), and any additional meta data. Pie charts are usually not all that useful in this kind of analysis; but one is attached based on Kyle's work. http://people.mozilla.org/~chofmann/installer/installer-pie.png Bar charts can be a bit more useful since its easier to see pct. contributed by each component and where the long tail of uninteresting/disjointed components starts to kick in. http://people.mozilla.org/~chofmann/installer/installer-bar.png The top 4 components make up the bulk of the package and then anything past the next 8 components is less that 1% of the package. 31.28% ./core/xul.dll 13.97% ./core/omni.ja 13.00% ./core/icudt52.dll 11.16% ./core/browser/omni.ja 6.10% ./core/gkmedias.dll 4.64% ./core/mozjs.dll 4.04% ./core/d3dcompiler_46.dll 2.63% ./core/D3DCompiler_43.dll 2.25% ./core/nss3.dll 1.28% ./core/icuin52.dll 1.12% ./core/uninstall/helper.exe 1.00% ./core/icuuc52.dll 0.96% ./core/msvcr100.dll 0.86% ./setup.exe 0.80% ./core/libGLESv2.dll 0.78% ./core/dictionaries/en-US.dic ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On Mon, Oct 13, 2014 at 5:52 PM, Gregory Szorc g...@mozilla.com wrote: On 10/13/14 5:42 PM, Andreas Gal wrote: I looked at lzma2 a while ago for FFOS. I got pretty consistently 30% smaller omni.ja with that. We could add it pretty easily to our decompression code but it has slightly different memory behavior. This was discussed in the Including Adobe CMaps thread (https://groups.google.com/d/msg/mozilla.dev.platform/SOInehcZa2g/ezcQRH7k_moJ). tl;dr is we need per-file compression on client for performance reasons: whole archive compression for actual Gecko load is a non-starter. I see now that glandium mentioned adding functionality to the installer in that thread. I guess it was never acted upon. I'm not sure if this is what's already proposed, but technically we could repackage the files during installation. I.e. repackage them from being optimized for size to being optimized for speed. This would definitely be a non-trivial amount of work though. But might be worth it if we got a 30% faster download. / Jonas ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 10/13/14 5:16 PM, Justin Dolske wrote: On 10/13/14 4:54 PM, Chris More wrote: Why am I asking this? The win32 Firefox full installer continues to grow (see attachment) each release and it has been on an increasing growth since Firefox 29. Like anything on the web, the time it takes to download something (webpage, binary file, etc.) affects the key conversion rate by some amount. We have the (much smaller) stub installer for Windows now... I know download size was a big factor for the full installer; do we know anything about kind of impact it has for the stub installer? Justin Sure the stub allows for some better response and user interaction in the early part of the process which can help sucess rates. At a minimum it least give us some visibility into what is really happening at the early stages of visiting the web site, and getting process kicked off, but still leaves holes in understand the back end of the process. There is another way to think about this. Its not so much a size problem we are working against, but a time problem. Of course size has impact on time, but its really time for the installer to complete the download and installation that is the big challenge. The longer the download takes the more chances there are for interruption and failure from network errors, user distractions and giving up, and other things that can get in the way of completion. Slow networks cause increase of time and more chance for failures. If we are serving installers now to a higher pct. of users in under developed world and slower connections we should expect the failure rates to be increasing, and if we are expanding the size of the download we are increasing the problem. I think john jensen studied this a bit and had some a/b studies where we increased the size of the installer and watched for failure rates to increase but don't recall where the results were put. I also think there were some serious limitations on the study and being able to measure significant changes in download size, and different failure rates at different download speeds/length of the installation process. -chofmann ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Breakdown of Firefox full installer
On 10/13/14 5:37 PM, Chris Hofmann wrote: and from last year Firefox installer size: How big is too big? - 2013 https://groups.google.com/forum/#!searchin/mozilla.dev.planning/installer$20size/mozilla.dev.planning/hPgUBzweL70/NeOjEf0hsh0J That thread about installer size was regarding the (controversial) inclusion of 3 MB of ICU data for SpiderMonkey's ECMA-402 i18n API. Chris More organized a funnelcake experiment (bug 933847) to inflate the Firefox 25 installer by 3 MB. The report's conclusions: * Increasing the download size from 22MB to 25MB had no statistical affect on conversion rates, so we landed ICU. * (Countries' average) Internet speed dramatically affected conversion rates: 70% success in the fastest countries and 30% in the slowest countries. Going forward, it would be interesting to see a dashboard track Firefox installer size every day (or show every changeset's delta on Treeherder). chris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
