RE: [QUESTION] How Many Pre-Built Binaries are Enough?

[Dennis E. Hamilton]

Thanks for looking into this.

There is an alternative to what Rory proposes on a different thread that does 
not require active integration on the part of a download server.  For security 
and authentication reasons I don't think off-Apache download integration that 
would be acceptable.

Depending on how language packs work, and ability to have multiple language 
packs installed, etc., another approach would be to provide a main distribution 
that carries a small number of language packs for selection among as part of 
the installation process.  Additional language packs could be downloadable 
almost as extensions, and even at install time if the primary language is other 
than one of the ones carried with the primary install.

Yes, that's speculative.  I do think the quantity and variation is the problem 
almost more than the size, since it is a choke point on the building of a full 
distribution of binaries, even for release candidates and their QA.  Smaller is 
still better -- it takes bandwidth on the part of our mirror sites, not just 
storage space.

It is also necessary to take into consideration third-party construction of 
distributions and how they might take advantage of a language-selection 
mechanism or not.  It would be useful to engage such distributors in this 
conversation.

Based on what the impact is on the deployment pipeline for release candidates 
and QA, it would be valuable to streamline and simplify the deployment process, 
including preparation of candidates.

Just pondering ...

I look forward to the results of experimentation in this area.

 - Dennis

PS: I agree that reducing the space for Linux distributions is important, but 
the bandwidth impact is primarily for Windows and then Macintosh and finally 
Linux.  And the release-related distribution pipeline issue for binaries is for 
all of them.  Am I missing something?

> -Original Message-
> From: Andrea Pescetti [mailto:pesce...@apache.org]
> Sent: Sunday, November 8, 2015 23:20
> To: dev@openoffice.apache.org
> Subject: Re: [QUESTION] How Many Pre-Built Binaries are Enough?
> 
> On 21/10/2015 Dennis E. Hamilton wrote:
> >4 flavors for Linux, taking 67%
> >1 flavor for MacOS, for 18%
> >1 flavor for Windows (win32 x86), for 15%. ...
> > when is it time to reduce those that represent inordinate demands to
> the needs for QA, distribution, and support?
> 
> The time is now. Not in terms of QA and support (we can cope with that),
> but in terms of packaging. A very significant bottleneck we have is the
> upload process to make binaries available: true, we are in 2015, but 40
> GBytes are still a big amount of data to move around. Uploading RC1 took
> more than four days of attempts; then things were streamlined with the
> help of Infra, but still very painful at times.
> 
> We need to reduce it to somewhere between 20 and 30 GBytes. We could be
> much more aggressive, but reducing to 20-30 GBytes would have resulted
> in several days saved when evaluating/testing the 3 RCs we made for
> OpenOffice 4.1.2.
> 
> The good thing is that, for Linux, it seems we can rearrange packages in
> a way that:
> 1) Does not require any changes to download scripts
> 2) Does not require major changes to the installation experience
> 3) Allows to reduce disk space for a full release by at least 10 GBytes
> 
> I didn't have time for completing the tests last weekend, but if this
> succeeds it will be worth evaluating. From a release management point of
> view, this is the only concern: creating binary packages for Linux-based
> systems, testing them and supporting them is covered. Disk space
> (actually, over-the-network file transfers) is the main issue.
> 
> Regards,
>Andrea.
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
> For additional commands, e-mail: dev-h...@openoffice.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
For additional commands, e-mail: dev-h...@openoffice.apache.org

Re: [QUESTION] How Many Pre-Built Binaries are Enough?

[Andrea Pescetti]

On 21/10/2015 Dennis E. Hamilton wrote:

   4 flavors for Linux, taking 67%
   1 flavor for MacOS, for 18%
   1 flavor for Windows (win32 x86), for 15%. ...
when is it time to reduce those that represent inordinate demands to the needs 
for QA, distribution, and support?


The time is now. Not in terms of QA and support (we can cope with that), 
but in terms of packaging. A very significant bottleneck we have is the 
upload process to make binaries available: true, we are in 2015, but 40 
GBytes are still a big amount of data to move around. Uploading RC1 took 
more than four days of attempts; then things were streamlined with the 
help of Infra, but still very painful at times.


We need to reduce it to somewhere between 20 and 30 GBytes. We could be 
much more aggressive, but reducing to 20-30 GBytes would have resulted 
in several days saved when evaluating/testing the 3 RCs we made for 
OpenOffice 4.1.2.


The good thing is that, for Linux, it seems we can rearrange packages in 
a way that:

1) Does not require any changes to download scripts
2) Does not require major changes to the installation experience
3) Allows to reduce disk space for a full release by at least 10 GBytes

I didn't have time for completing the tests last weekend, but if this 
succeeds it will be worth evaluating. From a release management point of 
view, this is the only concern: creating binary packages for Linux-based 
systems, testing them and supporting them is covered. Disk space 
(actually, over-the-network file transfers) is the main issue.


Regards,
  Andrea.

-
To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
For additional commands, e-mail: dev-h...@openoffice.apache.org

Re: [QUESTION] How Many Pre-Built Binaries are Enough?

[Kay Schenk]

On 10/21/2015 01:16 AM, Jürgen Schmidt wrote:
> On 21/10/15 09:55, Marcus wrote:
>> Am 10/21/2015 02:25 AM, schrieb Dennis E. Hamilton:
>>> I recently started a download of all of the AOO 4.1.2 rc2 candidates,
>>> then see how long it was going to take and let it run overnight.  In
>>> the morning, the download had failed to get them all.
>>>
>>> What struck me however is how big the downloads are.
>>>
>>> There are full pre-built, installable binaries for 41 separate
>>> languages, and separately-introducible smaller language packs for each
>>> of them as well.
>>>
>>> The pre-built binaries for each single language take 1s gigabyte, in
>>> six sets:
>>>
>>>4 flavors for Linux, taking 67%
>>>1 flavor for MacOS, for 18%
>>>1 flavor for Windows (win32 x86), for 15%.
>>>
>>> Yet the popularity of pre-built binaries, based on download
>>> statistics, is inverted, with about 88% for Windows, 9% for MacOS, and
>>> 3% for everything else.
>>>
>>> Now 3.2% (the actual fraction) of 41 million downloads of AOO 4.1.1 is
>>> still 1.3 million that include those Linux cases.
>>>
>>> QUESTION: Considering only pre-built binaries for downloading, my
>>> question is, when is it time to reduce those that represent inordinate
>>> demands to the needs for QA, distribution, and support?

So, which of these represent "inordinate demands"? Unless you're
saying that the space considerations for each platform are out of
line with the download percentages?

One subject we had barely broached in the past was the elimination
of the Linux-32 builds. We may want to revisit that. But, I think
the rpm and deb aspects are likely to remain. Maybe packaging Linux
could be improved?


>>>
>>> EXCLUDED:
>>>
>>>   1. This question is not about the relative use of pre-built full
>>> binaries for each of 41 languages.  There are other questions we could
>>> ask about that.  (The language packs for given languages are much
>>> smaller than the full binaries though, taking about 20MB per flavor.)
>>>
>>>   2. This question is not about the source codes and the SDK.  It is
>>> also not about the value of building and testing for different
>>> platforms.  It is only about distributing pre-built binaries.
>>>
>>> I suspect there are more details, and better questions, that could be
>>> raised.
>>>
>>> I'm all ears.  What are your concerns?  What do you see being
>>> overlooked in this calculus?
>>
>> in other words you are asking the question if and how the file sizes of
>> the install bits can be reduced? And when it is time to do this? Is this
>> right or have I understood something wrong?
>>
>> Thanks
>>
>> Marcus
>>
>>
> 
> no new question and discussion. This came up at the beginning at Apache
> and before. I remember that Oracle engineers worked on a new installer
> that had a multilingual install piece and download everything else on
> demand (or from a disc I think).
> 
> I believe we all agree that it doesn't make too much sense as it is
> today and a smarter packaging and installation would be nice.
> 
> The point is that many users rely on the localized version and can't
> really start with an English version only and install the necessary
> language pack. It's all about convenience of an end user product for the
> majority of our users.
> 
> A discussion if it make sense or not is useless, I would suggest to
> start a discussion on how to make it better from a technical perspective
> and work on a concept.
> 
> Juergen
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
> For additional commands, e-mail: dev-h...@openoffice.apache.org
> 

-- 

MzK

“The journey of a thousand miles begins
 with a single step.”
  --Lao Tzu



-
To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
For additional commands, e-mail: dev-h...@openoffice.apache.org

Re: [QUESTION] How Many Pre-Built Binaries are Enough?

[Marcus]

Am 10/21/2015 02:25 AM, schrieb Dennis E. Hamilton:

I recently started a download of all of the AOO 4.1.2 rc2 candidates, then see 
how long it was going to take and let it run overnight.  In the morning, the 
download had failed to get them all.

What struck me however is how big the downloads are.

There are full pre-built, installable binaries for 41 separate languages, and 
separately-introducible smaller language packs for each of them as well.

The pre-built binaries for each single language take 1s gigabyte, in six sets:

   4 flavors for Linux, taking 67%
   1 flavor for MacOS, for 18%
   1 flavor for Windows (win32 x86), for 15%.

Yet the popularity of pre-built binaries, based on download statistics, is 
inverted, with about 88% for Windows, 9% for MacOS, and 3% for everything else.

Now 3.2% (the actual fraction) of 41 million downloads of AOO 4.1.1 is still 
1.3 million that include those Linux cases.

QUESTION: Considering only pre-built binaries for downloading, my question is, 
when is it time to reduce those that represent inordinate demands to the needs 
for QA, distribution, and support?

EXCLUDED:

  1. This question is not about the relative use of pre-built full binaries for 
each of 41 languages.  There are other questions we could ask about that.  (The 
language packs for given languages are much smaller than the full binaries 
though, taking about 20MB per flavor.)

  2. This question is not about the source codes and the SDK.  It is also not 
about the value of building and testing for different platforms.  It is only 
about distributing pre-built binaries.

I suspect there are more details, and better questions, that could be raised.

I'm all ears.  What are your concerns?  What do you see being overlooked in 
this calculus?


in other words you are asking the question if and how the file sizes of 
the install bits can be reduced? And when it is time to do this? Is this 
right or have I understood something wrong?


Thanks

Marcus




-Original Message-
From: Dennis E. Hamilton [mailto:dennis.hamil...@acm.org]
Sent: Wednesday, September 23, 2015 18:38
To: dev@openoffice.apache.org
Subject: RE: [DISCUSS] Apache OpenOffice ODF in the Marketplace -
Downloading

Here is a rough, top-level view of Apache OpenOffice by the operating
systems it is downloaded for.  This should be no surprise.  To have some
grounding on the immediate situation, here are the downloading
statistics of Apache OpenOffice 4.1.1 so far.

 From Sourceforge,
.

Of the 41 million downloads that have occurred since release of 4.1.1,
until the end of August,

 87.7% are for Windows
  9.1% are for Macintosh
  3.2% is everything else, including Linux


[ ... ]


-
To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
For additional commands, e-mail: dev-h...@openoffice.apache.org

Re: [QUESTION] How Many Pre-Built Binaries are Enough?

[Jürgen Schmidt]

On 21/10/15 09:55, Marcus wrote:
> Am 10/21/2015 02:25 AM, schrieb Dennis E. Hamilton:
>> I recently started a download of all of the AOO 4.1.2 rc2 candidates,
>> then see how long it was going to take and let it run overnight.  In
>> the morning, the download had failed to get them all.
>>
>> What struck me however is how big the downloads are.
>>
>> There are full pre-built, installable binaries for 41 separate
>> languages, and separately-introducible smaller language packs for each
>> of them as well.
>>
>> The pre-built binaries for each single language take 1s gigabyte, in
>> six sets:
>>
>>4 flavors for Linux, taking 67%
>>1 flavor for MacOS, for 18%
>>1 flavor for Windows (win32 x86), for 15%.
>>
>> Yet the popularity of pre-built binaries, based on download
>> statistics, is inverted, with about 88% for Windows, 9% for MacOS, and
>> 3% for everything else.
>>
>> Now 3.2% (the actual fraction) of 41 million downloads of AOO 4.1.1 is
>> still 1.3 million that include those Linux cases.
>>
>> QUESTION: Considering only pre-built binaries for downloading, my
>> question is, when is it time to reduce those that represent inordinate
>> demands to the needs for QA, distribution, and support?
>>
>> EXCLUDED:
>>
>>   1. This question is not about the relative use of pre-built full
>> binaries for each of 41 languages.  There are other questions we could
>> ask about that.  (The language packs for given languages are much
>> smaller than the full binaries though, taking about 20MB per flavor.)
>>
>>   2. This question is not about the source codes and the SDK.  It is
>> also not about the value of building and testing for different
>> platforms.  It is only about distributing pre-built binaries.
>>
>> I suspect there are more details, and better questions, that could be
>> raised.
>>
>> I'm all ears.  What are your concerns?  What do you see being
>> overlooked in this calculus?
> 
> in other words you are asking the question if and how the file sizes of
> the install bits can be reduced? And when it is time to do this? Is this
> right or have I understood something wrong?
> 
> Thanks
> 
> Marcus
> 
> 

no new question and discussion. This came up at the beginning at Apache
and before. I remember that Oracle engineers worked on a new installer
that had a multilingual install piece and download everything else on
demand (or from a disc I think).

I believe we all agree that it doesn't make too much sense as it is
today and a smarter packaging and installation would be nice.

The point is that many users rely on the localized version and can't
really start with an English version only and install the necessary
language pack. It's all about convenience of an end user product for the
majority of our users.

A discussion if it make sense or not is useless, I would suggest to
start a discussion on how to make it better from a technical perspective
and work on a concept.

Juergen









-
To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
For additional commands, e-mail: dev-h...@openoffice.apache.org