Mirror operator contact email ( was: Sourceforge and AOO 3.4 distribution)
Howdy, NOTE - added infra ML on CC. Alright, a copy of the draft mail message (see below) was sent to each of the contact addresses for the 98 mirrors listed at http://www.openoffice.org/distribution/mirrors/ with the subject line of: 'OpenOffice.org to Apache OpenOffice mirror system migration' Of those 5 emails immediately bounce, 4 with 505 err and 1 with unk DNS err. There is another 8 or 9 entries on that page without an email contact listing, perhaps contact information can be found over the weekend. Just wanted to give the Infra team a heads up, I did not try to check for which if any of these are already ASF mirrors, wouldn't be surprised if there aren't a few...so who knows they just may contact the infa folks direct..if so, now @you know why, now. Hopefully this will lead to some added mirrors. Best wishes, //drew On Fri, 2012-03-23 at 13:47 -0400, drew wrote: On Wed, 2012-03-21 at 08:52 -0700, Joe Schaefer wrote: snip I still would like to strongly suggest that someone on the PPMC step up NOW and offer to contact existing mirror operators to get them to cover some of our losses. Howdy all, Please take a moment to review the draft of a contact email for direct mailing to the OO.o mirror operators. Dear OpenOffice.org Mirror Operator, On behalf of all the members of the OpenOffice.org community I would like to extend a warm thank you for the services rendered to the project while participating in the download mirror system. As you likely are aware, the OpenOffice.org project has during the last year migrated to the Apache Software Foundation (ASF) and taken the new name of Apache OpenOffice. Thanks to the efforts of numerous individuals contributing to the project I am pleased to say that the first full release of the Apache OpenOffice free open source (FOSS) office suite is nearing completion. The project is now looking for organizations that will again work with us to deliver this important linchpin of the FOSS ecosystem to literally millions of users worldwide. Specifically the project is looking for mirror operators to work within, and augment, the existing ASF server mirror system. Details regarding the ASF mirror system are available here: http://www.apache.org/info/how-to-mirror.html Please contact our project with any questions you may have via our public mailing list at ooo-dev@incubator.apache.org. In closing, please accept my thanks in advance for your time and consideration in this matter. I look forward to hearing from you and hope that our projects can once again work together to advance the cause of Free Open Source Software around the globe. Sincerely, Drew Jensen Apache OpenOffice (incubating) Poddling Project Management Committee atjen...@apache.org Comments, fixes and the like are very much appreciated. Given the current state of work I would like to get this out tonight if at all possible. Thanks //drew
Draft of mirror operator contact email ( was: Sourceforge and AOO 3.4 distribution)
On Wed, 2012-03-21 at 08:52 -0700, Joe Schaefer wrote: snip I still would like to strongly suggest that someone on the PPMC step up NOW and offer to contact existing mirror operators to get them to cover some of our losses. Howdy all, Please take a moment to review the draft of a contact email for direct mailing to the OO.o mirror operators. Dear OpenOffice.org Mirror Operator, On behalf of all the members of the OpenOffice.org community I would like to extend a warm thank you for the services rendered to the project while participating in the download mirror system. As you likely are aware, the OpenOffice.org project has during the last year migrated to the Apache Software Foundation (ASF) and taken the new name of Apache OpenOffice. Thanks to the efforts of numerous individuals contributing to the project I am pleased to say that the first full release of the Apache OpenOffice free open source (FOSS) office suite is nearing completion. The project is now looking for organizations that will again work with us to deliver this important linchpin of the FOSS ecosystem to literally millions of users worldwide. Specifically the project is looking for mirror operators to work within, and augment, the existing ASF server mirror system. Details regarding the ASF mirror system are available here: http://www.apache.org/info/how-to-mirror.html Please contact our project with any questions you may have via our public mailing list at ooo-dev@incubator.apache.org. In closing, please accept my thanks in advance for your time and consideration in this matter. I look forward to hearing from you and hope that our projects can once again work together to advance the cause of Free Open Source Software around the globe. Sincerely, Drew Jensen Apache OpenOffice (incubating) Poddling Project Management Committee atjen...@apache.org Comments, fixes and the like are very much appreciated. Given the current state of work I would like to get this out tonight if at all possible. Thanks //drew
Re: Draft of mirror operator contact email ( was: Sourceforge and AOO 3.4 distribution)
Lgtm Sent from my iPhone On Mar 23, 2012, at 1:47 PM, drew d...@baseanswers.com wrote: On Wed, 2012-03-21 at 08:52 -0700, Joe Schaefer wrote: snip I still would like to strongly suggest that someone on the PPMC step up NOW and offer to contact existing mirror operators to get them to cover some of our losses. Howdy all, Please take a moment to review the draft of a contact email for direct mailing to the OO.o mirror operators. Dear OpenOffice.org Mirror Operator, On behalf of all the members of the OpenOffice.org community I would like to extend a warm thank you for the services rendered to the project while participating in the download mirror system. As you likely are aware, the OpenOffice.org project has during the last year migrated to the Apache Software Foundation (ASF) and taken the new name of Apache OpenOffice. Thanks to the efforts of numerous individuals contributing to the project I am pleased to say that the first full release of the Apache OpenOffice free open source (FOSS) office suite is nearing completion. The project is now looking for organizations that will again work with us to deliver this important linchpin of the FOSS ecosystem to literally millions of users worldwide. Specifically the project is looking for mirror operators to work within, and augment, the existing ASF server mirror system. Details regarding the ASF mirror system are available here: http://www.apache.org/info/how-to-mirror.html Please contact our project with any questions you may have via our public mailing list at ooo-dev@incubator.apache.org. In closing, please accept my thanks in advance for your time and consideration in this matter. I look forward to hearing from you and hope that our projects can once again work together to advance the cause of Free Open Source Software around the globe. Sincerely, Drew Jensen Apache OpenOffice (incubating) Poddling Project Management Committee atjen...@apache.org Comments, fixes and the like are very much appreciated. Given the current state of work I would like to get this out tonight if at all possible. Thanks //drew
Re: Sourceforge and AOO 3.4 distribution
*We have finally assessed the capacity and capabilities needed to serve the surge of Apache OpenOffice 3.4 release-time traffic. Before we could commit to delivering the full download volume, we wanted to produce a vetted plan, including a clear timeline and backing technical implementation plans. First let me quickly recap my understanding of the problems we are trying to solve for: - Apache OpenOffice 3.4 will be released in mid April and we want to assure capacity to handle that traffic both in terms of bandwidth and simultaneous connections. - The Apache OpenOffice project would benefit to be able to promote the release heavily without worrying about capacity. Given those needs and the fact the Apache Infrastructure team said they’d welcome our assistance, we at SourceForge think we can help and that there would be mutual benefit. What we are proposing is an elaboration of Joe’s ‘hybrid’ approach: - Both AOO and SF.net mirror networks would be used to provide download capacity for the 3.4 release. - SourceForge.net would be the “recommended default download” on the website. - Apache Mirror network would be an alternate download option. - Apache OpenOffice team and Infrastructure team will maintain control of the the auto-update URL’s and possibly follow Rob’s suggestion to stagger automatic updates. SourceForge.net will manage the full burst capacity for web-based downloads through our global network of OSS mirrors, global CDN network(s) and cloud file server providers. Using these resources, we anticipate our capacity is well above the expected delivery requirements for the upcoming release. In addition to basic download capacity, SourceForge will provide detailed download statistics, which will support future product, infrastructure and marketing plans. We will commit to make stats available on the SourceForge.net website and provide stats delivery APIs. We are able to capture initiated downloads, not just page views, and will provide them split by geography and operating system. We’re also willing to consider additional stats needs. Proposed Timeline: - Immediately: SourceForge sets up Apache Infra team with credentials on an AOO mirror project in sf.net - First week: SourceForge updates contracts with CDN and other providers to handle full AOO peak release traffic - Second Week: AOO Infra team works with sf.net operations team to ramp traffic to sf.net in a controlled way in order to gather statistical data, verify assumptions, and give the Apache infrastrucure team time to verify our capacity. - 1-2 days post test: SF.net analyzes traffic data, assures that our assumptions about geographic mix, and interactive vs automated download mix, are valid and we can do this in a fiscally responsible way. - 1-2 days post test: AOO infrastructure team analyses traffic data, lets sf.net team know any additonal data needs, and validates that the system will work for them Once everything is tested and vetted on both sides, we will need to make a CDN bandwidth commit, and would like the AOO team to commit to notifying us 30 days prior to shutting down the flow of traffic, so that we can update our contracts and avoid penalties. We believe that the combination of SF.net mirrors, and CDN based burst capacity will provide a fast and stable download experience for AOO users, and **will allow the AOO team to publicize the release in an agressive manner.* On Wed, Mar 21, 2012 at 10:55 AM, Mark Ramm m...@geek.net wrote: And finally: would you have any objection to us using a mix of fixed mirrors, elastic file delivery services (like s3), and commercial CDN service to handle spikes in download gracefully and assure that global users get good download performance when local mirrors are overloaded or not available? No, we may even be willing to budget some amount for this purpose. Cost estimates would be appreciated as our budget numbers for FY2012 need to be finalized next week. Sorry that it's taken a bit to get back to you. We are working on getting pricing from a variety of providers, and my personal goal is to find a way for us to fund the CDN and S3 costs, and to provide this to the community as a free (as in beer) service. Thanks everybody who provided anecdotal information on historical traffic peaks, and particularly for the steady state run rate information. That has been invaluable as we talk with vendors about the suplemental capacity we need to acquire to handle peak loads. There's one key input to figuring out if I can pay for all of this out of ad revenue, which is what percentage of the daily downloads are expected to come from auto-updater software or other non-browser scripts? Would that traffic still be pointed primarily at AOO owned domains and mirrors, or would we be handling some of that from the sf.net service? And finally, I'd also be interested in finding out
Re: Sourceforge and AOO 3.4 distribution
And finally: would you have any objection to us using a mix of fixed mirrors, elastic file delivery services (like s3), and commercial CDN service to handle spikes in download gracefully and assure that global users get good download performance when local mirrors are overloaded or not available? No, we may even be willing to budget some amount for this purpose. Cost estimates would be appreciated as our budget numbers for FY2012 need to be finalized next week. Sorry that it's taken a bit to get back to you. We are working on getting pricing from a variety of providers, and my personal goal is to find a way for us to fund the CDN and S3 costs, and to provide this to the community as a free (as in beer) service. Thanks everybody who provided anecdotal information on historical traffic peaks, and particularly for the steady state run rate information. That has been invaluable as we talk with vendors about the suplemental capacity we need to acquire to handle peak loads. There's one key input to figuring out if I can pay for all of this out of ad revenue, which is what percentage of the daily downloads are expected to come from auto-updater software or other non-browser scripts? Would that traffic still be pointed primarily at AOO owned domains and mirrors, or would we be handling some of that from the sf.net service? And finally, I'd also be interested in finding out if you know percentage of traffic is from North America vs the rest of the world because some providers give very different rates for different locations, for example Cloudfront publishes $0.02/gb US and $0.12/gb in South America. Thanks again for to everybody who helped with data so far! --Mark Ramm This e- mail message is intended only for the named recipient(s) above. It may contain confidential and privileged information. If you are not the intended recipient you are hereby notified that any dissemination, distribution or copying of this e-mail and any attachment(s) is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender by replying to this e-mail and delete the message and any attachment(s) from your system. Thank you.
Re: Sourceforge and AOO 3.4 distribution
On Tue, 2012-03-20 at 15:45 -0400, Rob Weir wrote: On Tue, Mar 20, 2012 at 3:22 PM, Joe Schaefer joe_schae...@yahoo.com wrote: We're still exploring available options and collecting data imacat. Right now an existing ooo mirror operator has reported to us that his average bandwidth consumption for ooo was ~100Mbps. It would help us to know how many mirrors support the existing mirrorbrain service for ooo to get a guess as to what the impact would be for Apache mirrors, but we are anticipating similar bandwidth requirements for our mirrors given the available data. What we currently need are estimates related to peak downloads during the initial few days / weeks of a release. Anyone with historical data on this needs to step forward and share it ASAP- 300K strikes me as an off-peak figure at this point. I have not seen any actual log files with this info, but there are reported tidbits that might be useful, such as: OpenOffice.org 3.0 was downloaded 3 million times in its first week, with about 80% of the downloads by Windows users, an official with the group said in a blog post on Monday. http://www.computerworld.com/s/article/9117575/OpenOffice.org_3.0_scores_strong_first_week So per day that is 430,000, around 50% much more than the average we saw in February. Not as much as I expected. What I don't know is when they enabled the update notifications feature then, if it even existed in 3.0. I think that will have a big impact on download peaks. In fact, we might even want to be clever, like have a CGI that sometimes says there is an update available, and sometimes does not, just to spread out the load more evenly. For example, if we have our server respond you have the latest 90% of the time, then it will take several requests on average for the auto-update feature to prompt the user to download the update. So we have some ability to throttle that demand, based on our CGI. -Rob I thought that the mirror system had failed to keep up with demand in the first day or two of the 3.0 release, if that is true (I can't find a reference now), then it would of been higher if the servers had been able to keep up. Anyway my recall is that it was something about the download rush happening before the push from master to all the mirrors had finished - which IIRC was a recurring problem prompted by bloggers trying to scoop the release announcement. Anyway - just passing along what I recall //drew - Original Message - From: imacat ima...@mail.imacat.idv.tw To: Apache OpenOffice Developers ooo-dev@incubator.apache.org Cc: Sent: Tuesday, March 20, 2012 3:14 PM Subject: Re: Sourceforge and AOO 3.4 distribution Please correct me if I'm wrong. But I thought that the Apache mirrors have more bandwidth than the SourceForge mirrors. Could someone explain why do we put the default download to places with less bandwidth? Apache mirrors: http://www.apache.org/mirrors/ SourceForge mirrors: http://sourceforge.net/apps/trac/sourceforge/wiki/Mirrors On 2012/03/21 03:07, Dave Fisher said: On Mar 20, 2012, at 11:33 AM, Joe Schaefer wrote: From: Mark Ramm m...@geek.net To: ooo-dev@incubator.apache.org; Joe Schaefer joe_schae...@yahoo.com Cc: Ross Gardler rgard...@opendirective.com Sent: Tuesday, March 20, 2012 2:25 PM Subject: Re: Sourceforge and AOO 3.4 distribution On Mon, Mar 19, 2012 at 4:04 PM, Joe Schaefer joe_schae...@yahoo.com wrote: FWIW the ballpark figures we have today Roberto are roughly 12GB worth of release artifacts and about100TB / day worth of download traffic. Thanks for the information. I'm working with Roberto to make sure all the right technical resources are aligned behind him, and that we have the resources to provide a great experience to your users. So, I'm here to help out, and validate everything to make sure we are prepared to handle AOO's peak load. Based on the file size data in the previous e-mail, and this bandwidth information, I believe we are talking about something around 700k download per day. Is that peak load, or is that sustained load? If it's sustained, do you have any ideas about what peak load would look like? If not, do you have any ideas about what sustained load would look like? Up until the Update service broke last week, ooo was sustaining 300K downloads a day. We used a ballpark download figure of 300 MB per user, which may explain the discrepancy if you used something considerably less. We simply don't have any data at this point about peak load to make any educated guesses. When this subject came up last year Marcus described peak as 300,000 downloads / day. Stats were collected until last February's switch to Kenai. See http://www.openoffice.org/marketing/marketing_bouncer.html
Re: Sourceforge and AOO 3.4 distribution
Right now the ASF has roughly 250 mirrors, of which atm I'd expect 50 or so to drop us if we start carrying AOO releases. That means we'd have about twice as many mirrors as the mirrorbrain OOo service has, so we should be in better shape overall than what happened in the past, assuming we do something smart about staggering the Update service. There will be no way to surprise anybody about the release timing for an AOO release as at least a week of public discussion/voting will precede it. I still would like to strongly suggest that someone on the PPMC step up NOW and offer to contact existing mirror operators to get them to cover some of our losses. From: drew d...@baseanswers.com To: ooo-dev@incubator.apache.org Sent: Wednesday, March 21, 2012 11:45 AM Subject: Re: Sourceforge and AOO 3.4 distribution On Tue, 2012-03-20 at 15:45 -0400, Rob Weir wrote: On Tue, Mar 20, 2012 at 3:22 PM, Joe Schaefer joe_schae...@yahoo.com wrote: We're still exploring available options and collecting data imacat. Right now an existing ooo mirror operator has reported to us that his average bandwidth consumption for ooo was ~100Mbps. It would help us to know how many mirrors support the existing mirrorbrain service for ooo to get a guess as to what the impact would be for Apache mirrors, but we are anticipating similar bandwidth requirements for our mirrors given the available data. What we currently need are estimates related to peak downloads during the initial few days / weeks of a release. Anyone with historical data on this needs to step forward and share it ASAP- 300K strikes me as an off-peak figure at this point. I have not seen any actual log files with this info, but there are reported tidbits that might be useful, such as: OpenOffice.org 3.0 was downloaded 3 million times in its first week, with about 80% of the downloads by Windows users, an official with the group said in a blog post on Monday. http://www.computerworld.com/s/article/9117575/OpenOffice.org_3.0_scores_strong_first_week So per day that is 430,000, around 50% much more than the average we saw in February. Not as much as I expected. What I don't know is when they enabled the update notifications feature then, if it even existed in 3.0. I think that will have a big impact on download peaks. In fact, we might even want to be clever, like have a CGI that sometimes says there is an update available, and sometimes does not, just to spread out the load more evenly. For example, if we have our server respond you have the latest 90% of the time, then it will take several requests on average for the auto-update feature to prompt the user to download the update. So we have some ability to throttle that demand, based on our CGI. -Rob I thought that the mirror system had failed to keep up with demand in the first day or two of the 3.0 release, if that is true (I can't find a reference now), then it would of been higher if the servers had been able to keep up. Anyway my recall is that it was something about the download rush happening before the push from master to all the mirrors had finished - which IIRC was a recurring problem prompted by bloggers trying to scoop the release announcement. Anyway - just passing along what I recall //drew - Original Message - From: imacat ima...@mail.imacat.idv.tw To: Apache OpenOffice Developers ooo-dev@incubator.apache.org Cc: Sent: Tuesday, March 20, 2012 3:14 PM Subject: Re: Sourceforge and AOO 3.4 distribution Please correct me if I'm wrong. But I thought that the Apache mirrors have more bandwidth than the SourceForge mirrors. Could someone explain why do we put the default download to places with less bandwidth? Apache mirrors: http://www.apache.org/mirrors/ SourceForge mirrors: http://sourceforge.net/apps/trac/sourceforge/wiki/Mirrors On 2012/03/21 03:07, Dave Fisher said: On Mar 20, 2012, at 11:33 AM, Joe Schaefer wrote: From: Mark Ramm m...@geek.net To: ooo-dev@incubator.apache.org; Joe Schaefer joe_schae...@yahoo.com Cc: Ross Gardler rgard...@opendirective.com Sent: Tuesday, March 20, 2012 2:25 PM Subject: Re: Sourceforge and AOO 3.4 distribution On Mon, Mar 19, 2012 at 4:04 PM, Joe Schaefer joe_schae...@yahoo.com wrote: FWIW the ballpark figures we have today Roberto are roughly 12GB worth of release artifacts and about100TB / day worth of download traffic. Thanks for the information. I'm working with Roberto to make sure all the right technical resources are aligned behind him, and that we have the resources to provide a great experience to your users. So, I'm here to help out, and validate everything to make sure we are prepared to handle AOO's peak load. Based on the file size data in the previous e-mail, and this bandwidth
Re: Sourceforge and AOO 3.4 distribution
--- Mer 21/3/12, Joe Schaefer ha scritto: ... Right now the ASF has roughly 250 mirrors, of which atm I'd expect 50 or so to drop us if we start carrying AOO releases. That means we'd have about twice as many mirrors as the mirrorbrain OOo service has, so we should be in better shape overall than what happened in the past, assuming we do something smart about staggering the Update service. There will be no way to surprise anybody about the release timing for an AOO release as at least a week of public discussion/voting will precede it. FWIW, I am pretty sure the problem will be windows binaries: if we did a source-only release we probably wouldn't lose any mirrors. Perhaps there is some way to make the binary packages opt-out for the mirrors? Pedro. I still would like to strongly suggest that someone on the PPMC step up NOW and offer to contact existing mirror operators to get them to cover some of our losses.
Re: Sourceforge and AOO 3.4 distribution
On Wed, 2012-03-21 at 08:52 -0700, Joe Schaefer wrote: Right now the ASF has roughly 250 mirrors, of which atm I'd expect 50 or so to drop us if we start carrying AOO releases. That means we'd have about twice as many mirrors as the mirrorbrain OOo service has, so we should be in better shape overall than what happened in the past, assuming we do something smart about staggering the Update service. There will be no way to surprise anybody about the release timing for an AOO release as at least a week of public discussion/voting will precede it. I still would like to strongly suggest that someone on the PPMC step up NOW and offer to contact existing mirror operators to get them to cover some of our losses. Well, I'm looking this page: http://www.openoffice.org/distribution/mirrors/ @anyone: Is there a list with contact information (email addresses) available also? //drew snip
Re: Sourceforge and AOO 3.4 distribution
Please lets leave that sort of issue to infrastructure to worry about- what we do about mirrors that cannot support the AOO bandwidth needs is not a concern for AOO itself. From: Pedro Giffuni p...@apache.org To: ooo-dev@incubator.apache.org; Joe Schaefer joe_schae...@yahoo.com Sent: Wednesday, March 21, 2012 12:31 PM Subject: Re: Sourceforge and AOO 3.4 distribution --- Mer 21/3/12, Joe Schaefer ha scritto: ... Right now the ASF has roughly 250 mirrors, of which atm I'd expect 50 or so to drop us if we start carrying AOO releases. That means we'd have about twice as many mirrors as the mirrorbrain OOo service has, so we should be in better shape overall than what happened in the past, assuming we do something smart about staggering the Update service. There will be no way to surprise anybody about the release timing for an AOO release as at least a week of public discussion/voting will precede it. FWIW, I am pretty sure the problem will be windows binaries: if we did a source-only release we probably wouldn't lose any mirrors. Perhaps there is some way to make the binary packages opt-out for the mirrors? Pedro. I still would like to strongly suggest that someone on the PPMC step up NOW and offer to contact existing mirror operators to get them to cover some of our losses.
Re: Sourceforge and AOO 3.4 distribution
Might be possible for Andrew to pull up the subscriber list for the now-defunct mirr...@distribution.openoffice.org list. From: drew d...@baseanswers.com To: ooo-dev@incubator.apache.org ooo-dev@incubator.apache.org Sent: Wednesday, March 21, 2012 12:32 PM Subject: Re: Sourceforge and AOO 3.4 distribution On Wed, 2012-03-21 at 08:52 -0700, Joe Schaefer wrote: Right now the ASF has roughly 250 mirrors, of which atm I'd expect 50 or so to drop us if we start carrying AOO releases. That means we'd have about twice as many mirrors as the mirrorbrain OOo service has, so we should be in better shape overall than what happened in the past, assuming we do something smart about staggering the Update service. There will be no way to surprise anybody about the release timing for an AOO release as at least a week of public discussion/voting will precede it. I still would like to strongly suggest that someone on the PPMC step up NOW and offer to contact existing mirror operators to get them to cover some of our losses. Well, I'm looking this page: http://www.openoffice.org/distribution/mirrors/ @anyone: Is there a list with contact information (email addresses) available also? //drew snip
Re: Sourceforge and AOO 3.4 distribution
Hi Peter, Is there any way to contact the existing MirrorBrain hosts who support OpenOffice.org? Is there a mailing list we can send a note to? We would like to see if any of them would we able to help mirror the AOO 3.4 release as well, by being an ASF mirror as well. More info on mirror requirements is here: http://www.apache.org/info/how-to-mirror.html -Rob On Wed, Mar 21, 2012 at 11:52 AM, Joe Schaefer joe_schae...@yahoo.com wrote: Right now the ASF has roughly 250 mirrors, of which atm I'd expect 50 or so to drop us if we start carrying AOO releases. That means we'd have about twice as many mirrors as the mirrorbrain OOo service has, so we should be in better shape overall than what happened in the past, assuming we do something smart about staggering the Update service. There will be no way to surprise anybody about the release timing for an AOO release as at least a week of public discussion/voting will precede it. I still would like to strongly suggest that someone on the PPMC step up NOW and offer to contact existing mirror operators to get them to cover some of our losses. From: drew d...@baseanswers.com To: ooo-dev@incubator.apache.org Sent: Wednesday, March 21, 2012 11:45 AM Subject: Re: Sourceforge and AOO 3.4 distribution On Tue, 2012-03-20 at 15:45 -0400, Rob Weir wrote: On Tue, Mar 20, 2012 at 3:22 PM, Joe Schaefer joe_schae...@yahoo.com wrote: We're still exploring available options and collecting data imacat. Right now an existing ooo mirror operator has reported to us that his average bandwidth consumption for ooo was ~100Mbps. It would help us to know how many mirrors support the existing mirrorbrain service for ooo to get a guess as to what the impact would be for Apache mirrors, but we are anticipating similar bandwidth requirements for our mirrors given the available data. What we currently need are estimates related to peak downloads during the initial few days / weeks of a release. Anyone with historical data on this needs to step forward and share it ASAP- 300K strikes me as an off-peak figure at this point. I have not seen any actual log files with this info, but there are reported tidbits that might be useful, such as: OpenOffice.org 3.0 was downloaded 3 million times in its first week, with about 80% of the downloads by Windows users, an official with the group said in a blog post on Monday. http://www.computerworld.com/s/article/9117575/OpenOffice.org_3.0_scores_strong_first_week So per day that is 430,000, around 50% much more than the average we saw in February. Not as much as I expected. What I don't know is when they enabled the update notifications feature then, if it even existed in 3.0. I think that will have a big impact on download peaks. In fact, we might even want to be clever, like have a CGI that sometimes says there is an update available, and sometimes does not, just to spread out the load more evenly. For example, if we have our server respond you have the latest 90% of the time, then it will take several requests on average for the auto-update feature to prompt the user to download the update. So we have some ability to throttle that demand, based on our CGI. -Rob I thought that the mirror system had failed to keep up with demand in the first day or two of the 3.0 release, if that is true (I can't find a reference now), then it would of been higher if the servers had been able to keep up. Anyway my recall is that it was something about the download rush happening before the push from master to all the mirrors had finished - which IIRC was a recurring problem prompted by bloggers trying to scoop the release announcement. Anyway - just passing along what I recall //drew - Original Message - From: imacat ima...@mail.imacat.idv.tw To: Apache OpenOffice Developers ooo-dev@incubator.apache.org Cc: Sent: Tuesday, March 20, 2012 3:14 PM Subject: Re: Sourceforge and AOO 3.4 distribution Please correct me if I'm wrong. But I thought that the Apache mirrors have more bandwidth than the SourceForge mirrors. Could someone explain why do we put the default download to places with less bandwidth? Apache mirrors: http://www.apache.org/mirrors/ SourceForge mirrors: http://sourceforge.net/apps/trac/sourceforge/wiki/Mirrors On 2012/03/21 03:07, Dave Fisher said: On Mar 20, 2012, at 11:33 AM, Joe Schaefer wrote: From: Mark Ramm m...@geek.net To: ooo-dev@incubator.apache.org; Joe Schaefer joe_schae...@yahoo.com Cc: Ross Gardler rgard...@opendirective.com Sent: Tuesday, March 20, 2012 2:25 PM Subject: Re: Sourceforge and AOO 3.4 distribution On Mon, Mar 19, 2012 at 4:04 PM, Joe Schaefer joe_schae...@yahoo.com wrote: FWIW the ballpark figures we have today Roberto are roughly 12GB worth of release artifacts
Re: Sourceforge and AOO 3.4 distribution
On Wed, 2012-03-21 at 12:32 -0400, drew wrote: On Wed, 2012-03-21 at 08:52 -0700, Joe Schaefer wrote: Right now the ASF has roughly 250 mirrors, of which atm I'd expect 50 or so to drop us if we start carrying AOO releases. That means we'd have about twice as many mirrors as the mirrorbrain OOo service has, so we should be in better shape overall than what happened in the past, assuming we do something smart about staggering the Update service. There will be no way to surprise anybody about the release timing for an AOO release as at least a week of public discussion/voting will precede it. I still would like to strongly suggest that someone on the PPMC step up NOW and offer to contact existing mirror operators to get them to cover some of our losses. Well, I'm looking this page: http://www.openoffice.org/distribution/mirrors/ @anyone: Is there a list with contact information (email addresses) available also? Never mind - the info is on that page, it was just an overflow column on my display.. OK - I'll work on an email today. //drew snip
Re: Sourceforge and AOO 3.4 distribution
Hah, I missed it as well ;-) From: drew d...@baseanswers.com To: ooo-dev@incubator.apache.org Sent: Wednesday, March 21, 2012 1:11 PM Subject: Re: Sourceforge and AOO 3.4 distribution On Wed, 2012-03-21 at 12:32 -0400, drew wrote: On Wed, 2012-03-21 at 08:52 -0700, Joe Schaefer wrote: Right now the ASF has roughly 250 mirrors, of which atm I'd expect 50 or so to drop us if we start carrying AOO releases. That means we'd have about twice as many mirrors as the mirrorbrain OOo service has, so we should be in better shape overall than what happened in the past, assuming we do something smart about staggering the Update service. There will be no way to surprise anybody about the release timing for an AOO release as at least a week of public discussion/voting will precede it. I still would like to strongly suggest that someone on the PPMC step up NOW and offer to contact existing mirror operators to get them to cover some of our losses. Well, I'm looking this page: http://www.openoffice.org/distribution/mirrors/ @anyone: Is there a list with contact information (email addresses) available also? Never mind - the info is on that page, it was just an overflow column on my display.. OK - I'll work on an email today. //drew snip
Re: Sourceforge and AOO 3.4 distribution
On Mon, Mar 19, 2012 at 4:04 PM, Joe Schaefer joe_schae...@yahoo.com wrote: FWIW the ballpark figures we have today Roberto are roughly 12GB worth of release artifacts and about100TB / day worth of download traffic. Thanks for the information. I'm working with Roberto to make sure all the right technical resources are aligned behind him, and that we have the resources to provide a great experience to your users. So, I'm here to help out, and validate everything to make sure we are prepared to handle AOO's peak load. Based on the file size data in the previous e-mail, and this bandwidth information, I believe we are talking about something around 700k download per day. Is that peak load, or is that sustained load? If it's sustained, do you have any ideas about what peak load would look like? If not, do you have any ideas about what sustained load would look like? And finally: would you have any objection to us using a mix of fixed mirrors, elastic file delivery services (like s3), and commercial CDN service to handle spikes in download gracefully and assure that global users get good download performance when local mirrors are overloaded or not available? I'm looking forward to working with all of you to make sure that users have a reliable and fast download source for the upcoming Apache Open Office release. Let me know if there's any questions I can answer for you, or anything else I can do to help. --Mark Ramm This e- mail message is intended only for the named recipient(s) above. It may contain confidential and privileged information. If you are not the intended recipient you are hereby notified that any dissemination, distribution or copying of this e-mail and any attachment(s) is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender by replying to this e-mail and delete the message and any attachment(s) from your system. Thank you.
Re: Sourceforge and AOO 3.4 distribution
From: Mark Ramm m...@geek.net To: ooo-dev@incubator.apache.org; Joe Schaefer joe_schae...@yahoo.com Cc: Ross Gardler rgard...@opendirective.com Sent: Tuesday, March 20, 2012 2:25 PM Subject: Re: Sourceforge and AOO 3.4 distribution On Mon, Mar 19, 2012 at 4:04 PM, Joe Schaefer joe_schae...@yahoo.com wrote: FWIW the ballpark figures we have today Roberto are roughly 12GB worth of release artifacts and about100TB / day worth of download traffic. Thanks for the information. I'm working with Roberto to make sure all the right technical resources are aligned behind him, and that we have the resources to provide a great experience to your users. So, I'm here to help out, and validate everything to make sure we are prepared to handle AOO's peak load. Based on the file size data in the previous e-mail, and this bandwidth information, I believe we are talking about something around 700k download per day. Is that peak load, or is that sustained load? If it's sustained, do you have any ideas about what peak load would look like? If not, do you have any ideas about what sustained load would look like? Up until the Update service broke last week, ooo was sustaining 300K downloads a day. We used a ballpark download figure of 300 MB per user, which may explain the discrepancy if you used something considerably less. We simply don't have any data at this point about peak load to make any educated guesses. And finally: would you have any objection to us using a mix of fixed mirrors, elastic file delivery services (like s3), and commercial CDN service to handle spikes in download gracefully and assure that global users get good download performance when local mirrors are overloaded or not available? No, we may even be willing to budget some amount for this purpose. Cost estimates would be appreciated as our budget numbers for FY2012 need to be finalized next week. I'm looking forward to working with all of you to make sure that users have a reliable and fast download source for the upcoming Apache Open Office release. Let me know if there's any questions I can answer for you, or anything else I can do to help. --Mark Ramm This e- mail message is intended only for the named recipient(s) above. It may contain confidential and privileged information. If you are not the intended recipient you are hereby notified that any dissemination, distribution or copying of this e-mail and any attachment(s) is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender by replying to this e-mail and delete the message and any attachment(s) from your system. Thank you.
Re: Sourceforge and AOO 3.4 distribution
On Mar 20, 2012, at 11:33 AM, Joe Schaefer wrote: From: Mark Ramm m...@geek.net To: ooo-dev@incubator.apache.org; Joe Schaefer joe_schae...@yahoo.com Cc: Ross Gardler rgard...@opendirective.com Sent: Tuesday, March 20, 2012 2:25 PM Subject: Re: Sourceforge and AOO 3.4 distribution On Mon, Mar 19, 2012 at 4:04 PM, Joe Schaefer joe_schae...@yahoo.com wrote: FWIW the ballpark figures we have today Roberto are roughly 12GB worth of release artifacts and about100TB / day worth of download traffic. Thanks for the information. I'm working with Roberto to make sure all the right technical resources are aligned behind him, and that we have the resources to provide a great experience to your users. So, I'm here to help out, and validate everything to make sure we are prepared to handle AOO's peak load. Based on the file size data in the previous e-mail, and this bandwidth information, I believe we are talking about something around 700k download per day. Is that peak load, or is that sustained load? If it's sustained, do you have any ideas about what peak load would look like? If not, do you have any ideas about what sustained load would look like? Up until the Update service broke last week, ooo was sustaining 300K downloads a day. We used a ballpark download figure of 300 MB per user, which may explain the discrepancy if you used something considerably less. We simply don't have any data at this point about peak load to make any educated guesses. When this subject came up last year Marcus described peak as 300,000 downloads / day. Stats were collected until last February's switch to Kenai. See http://www.openoffice.org/marketing/marketing_bouncer.html Depending on how we handle the announcement of AOO 3.4 - press, update service, and ooo-announce we might be able to spread a single spike into more smaller peaks. HTH, Dave And finally: would you have any objection to us using a mix of fixed mirrors, elastic file delivery services (like s3), and commercial CDN service to handle spikes in download gracefully and assure that global users get good download performance when local mirrors are overloaded or not available? No, we may even be willing to budget some amount for this purpose. Cost estimates would be appreciated as our budget numbers for FY2012 need to be finalized next week. I'm looking forward to working with all of you to make sure that users have a reliable and fast download source for the upcoming Apache Open Office release. Let me know if there's any questions I can answer for you, or anything else I can do to help. --Mark Ramm This e- mail message is intended only for the named recipient(s) above. It may contain confidential and privileged information. If you are not the intended recipient you are hereby notified that any dissemination, distribution or copying of this e-mail and any attachment(s) is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender by replying to this e-mail and delete the message and any attachment(s) from your system. Thank you.
Re: Sourceforge and AOO 3.4 distribution
Please correct me if I'm wrong. But I thought that the Apache mirrors have more bandwidth than the SourceForge mirrors. Could someone explain why do we put the default download to places with less bandwidth? Apache mirrors: http://www.apache.org/mirrors/ SourceForge mirrors: http://sourceforge.net/apps/trac/sourceforge/wiki/Mirrors On 2012/03/21 03:07, Dave Fisher said: On Mar 20, 2012, at 11:33 AM, Joe Schaefer wrote: From: Mark Ramm m...@geek.net To: ooo-dev@incubator.apache.org; Joe Schaefer joe_schae...@yahoo.com Cc: Ross Gardler rgard...@opendirective.com Sent: Tuesday, March 20, 2012 2:25 PM Subject: Re: Sourceforge and AOO 3.4 distribution On Mon, Mar 19, 2012 at 4:04 PM, Joe Schaefer joe_schae...@yahoo.com wrote: FWIW the ballpark figures we have today Roberto are roughly 12GB worth of release artifacts and about100TB / day worth of download traffic. Thanks for the information. I'm working with Roberto to make sure all the right technical resources are aligned behind him, and that we have the resources to provide a great experience to your users. So, I'm here to help out, and validate everything to make sure we are prepared to handle AOO's peak load. Based on the file size data in the previous e-mail, and this bandwidth information, I believe we are talking about something around 700k download per day. Is that peak load, or is that sustained load? If it's sustained, do you have any ideas about what peak load would look like? If not, do you have any ideas about what sustained load would look like? Up until the Update service broke last week, ooo was sustaining 300K downloads a day. We used a ballpark download figure of 300 MB per user, which may explain the discrepancy if you used something considerably less. We simply don't have any data at this point about peak load to make any educated guesses. When this subject came up last year Marcus described peak as 300,000 downloads / day. Stats were collected until last February's switch to Kenai. See http://www.openoffice.org/marketing/marketing_bouncer.html Depending on how we handle the announcement of AOO 3.4 - press, update service, and ooo-announce we might be able to spread a single spike into more smaller peaks. HTH, Dave And finally: would you have any objection to us using a mix of fixed mirrors, elastic file delivery services (like s3), and commercial CDN service to handle spikes in download gracefully and assure that global users get good download performance when local mirrors are overloaded or not available? No, we may even be willing to budget some amount for this purpose. Cost estimates would be appreciated as our budget numbers for FY2012 need to be finalized next week. I'm looking forward to working with all of you to make sure that users have a reliable and fast download source for the upcoming Apache Open Office release. Let me know if there's any questions I can answer for you, or anything else I can do to help. --Mark Ramm This e- mail message is intended only for the named recipient(s) above. It may contain confidential and privileged information. If you are not the intended recipient you are hereby notified that any dissemination, distribution or copying of this e-mail and any attachment(s) is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender by replying to this e-mail and delete the message and any attachment(s) from your system. Thank you. -- Best regards, imacat ^_*' ima...@mail.imacat.idv.tw PGP Key http://www.imacat.idv.tw/me/pgpkey.asc Woman's Voice News: http://www.wov.idv.tw/ Tavern IMACAT's http://www.imacat.idv.tw/ Woman in FOSS in Taiwan http://wofoss.blogspot.com/ Apache OpenOffice http://www.openoffice.org/ EducOO/OOo4Kids Taiwan http://www.educoo.tw/ signature.asc Description: OpenPGP digital signature
Re: Sourceforge and AOO 3.4 distribution
We're still exploring available options and collecting data imacat. Right now an existing ooo mirror operator has reported to us that his average bandwidth consumption for ooo was ~100Mbps. It would help us to know how many mirrors support the existing mirrorbrain service for ooo to get a guess as to what the impact would be for Apache mirrors, but we are anticipating similar bandwidth requirements for our mirrors given the available data. What we currently need are estimates related to peak downloads during the initial few days / weeks of a release. Anyone with historical data on this needs to step forward and share it ASAP- 300K strikes me as an off-peak figure at this point. - Original Message - From: imacat ima...@mail.imacat.idv.tw To: Apache OpenOffice Developers ooo-dev@incubator.apache.org Cc: Sent: Tuesday, March 20, 2012 3:14 PM Subject: Re: Sourceforge and AOO 3.4 distribution Please correct me if I'm wrong. But I thought that the Apache mirrors have more bandwidth than the SourceForge mirrors. Could someone explain why do we put the default download to places with less bandwidth? Apache mirrors: http://www.apache.org/mirrors/ SourceForge mirrors: http://sourceforge.net/apps/trac/sourceforge/wiki/Mirrors On 2012/03/21 03:07, Dave Fisher said: On Mar 20, 2012, at 11:33 AM, Joe Schaefer wrote: From: Mark Ramm m...@geek.net To: ooo-dev@incubator.apache.org; Joe Schaefer joe_schae...@yahoo.com Cc: Ross Gardler rgard...@opendirective.com Sent: Tuesday, March 20, 2012 2:25 PM Subject: Re: Sourceforge and AOO 3.4 distribution On Mon, Mar 19, 2012 at 4:04 PM, Joe Schaefer joe_schae...@yahoo.com wrote: FWIW the ballpark figures we have today Roberto are roughly 12GB worth of release artifacts and about100TB / day worth of download traffic. Thanks for the information. I'm working with Roberto to make sure all the right technical resources are aligned behind him, and that we have the resources to provide a great experience to your users. So, I'm here to help out, and validate everything to make sure we are prepared to handle AOO's peak load. Based on the file size data in the previous e-mail, and this bandwidth information, I believe we are talking about something around 700k download per day. Is that peak load, or is that sustained load? If it's sustained, do you have any ideas about what peak load would look like? If not, do you have any ideas about what sustained load would look like? Up until the Update service broke last week, ooo was sustaining 300K downloads a day. We used a ballpark download figure of 300 MB per user, which may explain the discrepancy if you used something considerably less. We simply don't have any data at this point about peak load to make any educated guesses. When this subject came up last year Marcus described peak as 300,000 downloads / day. Stats were collected until last February's switch to Kenai. See http://www.openoffice.org/marketing/marketing_bouncer.html Depending on how we handle the announcement of AOO 3.4 - press, update service, and ooo-announce we might be able to spread a single spike into more smaller peaks. HTH, Dave And finally: would you have any objection to us using a mix of fixed mirrors, elastic file delivery services (like s3), and commercial CDN service to handle spikes in download gracefully and assure that global users get good download performance when local mirrors are overloaded or not available? No, we may even be willing to budget some amount for this purpose. Cost estimates would be appreciated as our budget numbers for FY2012 need to be finalized next week. I'm looking forward to working with all of you to make sure that users have a reliable and fast download source for the upcoming Apache Open Office release. Let me know if there's any questions I can answer for you, or anything else I can do to help. --Mark Ramm This e- mail message is intended only for the named recipient(s) above. It may contain confidential and privileged information. If you are not the intended recipient you are hereby notified that any dissemination, distribution or copying of this e-mail and any attachment(s) is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender by replying to this e-mail and delete the message and any attachment(s) from your system. Thank you. -- Best regards, imacat ^_*' ima...@mail.imacat.idv.tw PGP Key http://www.imacat.idv.tw/me/pgpkey.asc Woman's Voice News: http://www.wov.idv.tw/ Tavern IMACAT's http://www.imacat.idv.tw/ Woman in FOSS in Taiwan http://wofoss.blogspot.com/ Apache OpenOffice http://www.openoffice.org/ EducOO/OOo4Kids Taiwan http://www.educoo.tw/
Re: Sourceforge and AOO 3.4 distribution
Am 03/20/2012 08:07 PM, schrieb Dave Fisher: On Mar 20, 2012, at 11:33 AM, Joe Schaefer wrote: From: Mark Rammm...@geek.net To: ooo-dev@incubator.apache.org; Joe Schaeferjoe_schae...@yahoo.com Cc: Ross Gardlerrgard...@opendirective.com Sent: Tuesday, March 20, 2012 2:25 PM Subject: Re: Sourceforge and AOO 3.4 distribution On Mon, Mar 19, 2012 at 4:04 PM, Joe Schaeferjoe_schae...@yahoo.com wrote: FWIW the ballpark figures we have today Roberto are roughly 12GB worth of release artifacts and about100TB / day worth of download traffic. Thanks for the information. I'm working with Roberto to make sure all the right technical resources are aligned behind him, and that we have the resources to provide a great experience to your users. So, I'm here to help out, and validate everything to make sure we are prepared to handle AOO's peak load. Based on the file size data in the previous e-mail, and this bandwidth information, I believe we are talking about something around 700k download per day. Is that peak load, or is that sustained load? If it's sustained, do you have any ideas about what peak load would look like? If not, do you have any ideas about what sustained load would look like? Up until the Update service broke last week, ooo was sustaining 300K downloads a day. We used a ballpark download figure of 300 MB per user, which may explain the discrepancy if you used something considerably less. We simply don't have any data at this point about peak load to make any educated guesses. When this subject came up last year Marcus described peak as 300,000 downloads / day. Yes, 300k / day was the number we have counted with. But the good news is, it's not the peak number but average. For new releases I don't remember any peak numbers. But I absolutely would not be surprised when it comes to a few millions for the first days of the new release. That should decrease considerable after ~1 week. However, this was at Sun/Oracle times with known surprises. Now at Apache it's a kind of new game. ;-) What comes into my mind right now, I *highly recommend* to let the update service response with a message like You have the most recent version when we have officially announced the AOO 3.4 release in the public. This announcement alone (and the countless copies in the news portals) will result in a high download peak. So, with an enabled update service telling We have new bits for you *together* with the annoucement would not be clever. The best is to do it 1-2 weeks after the announcement when we see that the peaks are decrease considerable. That's what we have done at Sun/Oracle and it was a good decision to let the mirrors survive the first peak downloads. Stats were collected until last February's switch to Kenai. See http://www.openoffice.org/marketing/marketing_bouncer.html Even when these numbers are from Feb 2010 it should be very similar for 2011. Marcus Depending on how we handle the announcement of AOO 3.4 - press, update service, and ooo-announce we might be able to spread a single spike into more smaller peaks. HTH, Dave And finally: would you have any objection to us using a mix of fixed mirrors, elastic file delivery services (like s3), and commercial CDN service to handle spikes in download gracefully and assure that global users get good download performance when local mirrors are overloaded or not available? No, we may even be willing to budget some amount for this purpose. Cost estimates would be appreciated as our budget numbers for FY2012 need to be finalized next week. I'm looking forward to working with all of you to make sure that users have a reliable and fast download source for the upcoming Apache Open Office release. Let me know if there's any questions I can answer for you, or anything else I can do to help. --Mark Ramm This e- mail message is intended only for the named recipient(s) above. It may contain confidential and privileged information. If you are not the intended recipient you are hereby notified that any dissemination, distribution or copying of this e-mail and any attachment(s) is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender by replying to this e-mail and delete the message and any attachment(s) from your system. Thank you.
Sourceforge and AOO 3.4 distribution
I just had a call with Roberto from SourceForge in which he updated me on what they've done with the templates and extensions sites. I asked Roberto to send a summary to this list, but I just wanted to extend my thanks to him and the team at SourceForge, along with the people here in AOO and ASF infra who have helped. Roberto also asked if there is anything SF can do to help distributing the AOO 3.4 We've discussed this a few times but as we are now close to a release I think it is worth recapping and making sure everything is lined up OK. - what are the likely bandwidth requirements when the release goes out? - does ASF Infra feel confident the existing mirror network will support this? - can SF become a part of that mirror network in a sensible way? - note that SF does not provide direct links to the download, they provide an intermediate page with advertising - should any of the old OOo mirrors be incorporated into the ASF mirror system? - will they want the additional overhead brought by all other ASF projects? My own feeling is that the ASF infra team would not really be interesting in changing the mirror system in any way, however I am *not* member of the infra team and cannot speak for them. Joe, of course is. If the PPMC sees the need to explore SF hosting then I suggest someone picks this up and liaises between ASF Infra, AOO and SF. IF the PPMC is confident that the existing mirror system is sufficient then no need to revisit. Ross -- Ross Gardler (@rgardler) Programme Leader (Open Development) OpenDirective http://opendirective.com
Re: Sourceforge and AOO 3.4 distribution
On 2012/03/19 23:03, Ross Gardler said: - what are the likely bandwidth requirements when the release goes out? - does ASF Infra feel confident the existing mirror network will support this? - can SF become a part of that mirror network in a sensible way? - note that SF does not provide direct links to the download, they provide an intermediate page with advertising - should any of the old OOo mirrors be incorporated into the ASF mirror system? - will they want the additional overhead brought by all other ASF projects? My own feeling is that the ASF infra team would not really be interesting in changing the mirror system in any way, however I am *not* member of the infra team and cannot speak for them. Joe, of Speaking of this, I am also interested to know the mirror system when we are going to distribute AOO 3.4. If we miss local mirrors in some countries, do we need to add them? -- Best regards, imacat ^_*' ima...@mail.imacat.idv.tw PGP Key http://www.imacat.idv.tw/me/pgpkey.asc Woman's Voice News: http://www.wov.idv.tw/ Tavern IMACAT's http://www.imacat.idv.tw/ Woman in FOSS in Taiwan http://wofoss.blogspot.com/ Apache OpenOffice http://www.openoffice.org/ EducOO/OOo4Kids Taiwan http://www.educoo.tw/ signature.asc Description: OpenPGP digital signature
Re: Sourceforge and AOO 3.4 distribution
- Original Message - From: Ross Gardler rgard...@opendirective.com To: ooo-dev@incubator.apache.org Cc: Sent: Monday, March 19, 2012 11:03 AM Subject: Sourceforge and AOO 3.4 distribution I just had a call with Roberto from SourceForge in which he updated me on what they've done with the templates and extensions sites. I asked Roberto to send a summary to this list, but I just wanted to extend my thanks to him and the team at SourceForge, along with the people here in AOO and ASF infra who have helped. Roberto also asked if there is anything SF can do to help distributing the AOO 3.4 We've discussed this a few times but as we are now close to a release I think it is worth recapping and making sure everything is lined up OK. - what are the likely bandwidth requirements when the release goes out? As far as Infra is concerned, it will depend on the total size of the artifacts being released multiplied by the number of mirrors that need to download it from us over a 6 hour period. We are considering rate-limiting our rsync service to lower the peak bandwidth needed. - does ASF Infra feel confident the existing mirror network will support this? I'd say most mirrors won't object once we give them a heads-up about how much additional disk space and bandwidth will be required. It would help if the PPMC could provide infra with an estimate of the expected number of total downloads per day during the first week or two of release, combined with the typical download size, so we may provide that information to the mirror operators and let them decide whether to stay with us or drop out. - can SF become a part of that mirror network in a sensible way? - note that SF does not provide direct links to the download, they provide an intermediate page with advertising The advertising does not exactly thrill me, and really isn't compatible with how our mirror scripts work. - should any of the old OOo mirrors be incorporated into the ASF mirror system? - will they want the additional overhead brought by all other ASF projects? My own feeling is that the ASF infra team would not really be interesting in changing the mirror system in any way, however I am *not* member of the infra team and cannot speak for them. Joe, of course is. If the PPMC sees the need to explore SF hosting then I suggest someone picks this up and liaises between ASF Infra, AOO and SF. IF the PPMC is confident that the existing mirror system is sufficient then no need to revisit. Ross -- Ross Gardler (@rgardler) Programme Leader (Open Development) OpenDirective http://opendirective.com
Re: Sourceforge and AOO 3.4 distribution
On Mon, Mar 19, 2012 at 11:03 AM, Ross Gardler rgard...@opendirective.com wrote: I just had a call with Roberto from SourceForge in which he updated me on what they've done with the templates and extensions sites. I asked Roberto to send a summary to this list, but I just wanted to extend my thanks to him and the team at SourceForge, along with the people here in AOO and ASF infra who have helped. Roberto also asked if there is anything SF can do to help distributing the AOO 3.4 We've discussed this a few times but as we are now close to a release I think it is worth recapping and making sure everything is lined up OK. - what are the likely bandwidth requirements when the release goes out? - does ASF Infra feel confident the existing mirror network will support this? There is a saying among performance engineers, that a man can drown in a stream that is 2 feet deep on average. It is critical here to look at peak demand, both in time as well as geographically. The geo aspect is interesting. Traditionally OOo had heavy usage in countries such as Brazil, South Africa, Australia, New Zealand, etc. It would be interesting to see if the Apache mirror density in these areas is similar to what we had for OOo. - can SF become a part of that mirror network in a sensible way? - note that SF does not provide direct links to the download, they provide an intermediate page with advertising - should any of the old OOo mirrors be incorporated into the ASF mirror system? - will they want the additional overhead brought by all other ASF projects? My own feeling is that the ASF infra team would not really be interesting in changing the mirror system in any way, however I am *not* member of the infra team and cannot speak for them. Joe, of course is. If the PPMC sees the need to explore SF hosting then I suggest someone picks this up and liaises between ASF Infra, AOO and SF. IF the PPMC is confident that the existing mirror system is sufficient then no need to revisit. Ross -- Ross Gardler (@rgardler) Programme Leader (Open Development) OpenDirective http://opendirective.com
Re: Sourceforge and AOO 3.4 distribution
Hi Ross, On 3/19/12 4:03 PM, Ross Gardler wrote: I just had a call with Roberto from SourceForge in which he updated me on what they've done with the templates and extensions sites. I asked Roberto to send a summary to this list, but I just wanted to extend my thanks to him and the team at SourceForge, along with the people here in AOO and ASF infra who have helped. Roberto also asked if there is anything SF can do to help distributing the AOO 3.4 We've discussed this a few times but as we are now close to a release I think it is worth recapping and making sure everything is lined up OK. good timing, I just send out an email today to infrastructure-dev to ask what kind of preparation we need for our upcoming release. In a first response I got some feedback that the required (and expected growing) size will probably a problem in the future. I prepared also some rough estimation based on 4 platforms and 13 languages (similar to the provided dev snapshots) See form example: Platforms = We plan to release Windows, Linux, MacOS FreeBSD is handled separately directly from the BSD guys Solaris can be seen as a further possible candidate in the future. Products AOO is a end-user oriented product and the user experience is important. That means we would like to provide language specific binary builds for our users to provide the same comfort and user experience as before. Languages = AOO was available in ~100 languages and it would be nice if we can address as much as possible but for the beginning we would at least start with languages where we got feedback and/or new translations for. en-US de fr es it ja zh-CN pt-BR nl ru hu fi zh-TW Probably more language packs when we have the translations ready Some numbers about the required space = A first rough estimation about the required space. We would need round about ~12GB for a proposed release for the 4 platforms (Windows, Linux (x86, x86-64), MacOS) and 13 languages. It can grow quite fast if we want release Solaris as well and more languages. Source Releases === aoo-3.4-src.zip = ~373MB aoo-3.4-src.tar.gz = ~312MB aoo-3.4-src.tar.bz2 = ~250MB asc, md5, sha1, sha512 files = ~3K Size = ~935MB MacOS = dmg = ~160MB number of languages = 13 asc, md5, sha1, sha512 files = ~10K Size = 13 * 160MB = 1.69GB Windows === download exe = 109MB number of languages = 13 asc, md5, sha1, sha512 files = ~10K (not verified yet) Size = 13 * 109MB = 1.417GB Linux = x86 rpm.tar.gz = 139MB deb.tar.gz = 138MB number of languages = 13 asc, md5, sha1, sha512 files = ~20K ((not verified yet)) Size = 13 * 138MB + 9 * 139MB = ~3.6GB x86-64 rpm.tar.gz = 148MB deb.tar.gz = 147MB number of languages = 13 asc, md5, sha1, sha512 files = ~20K (not verified yet) Size = 13 * 148MB + 9 * 147MB = ~3.835GB - what are the likely bandwidth requirements when the release goes out? - does ASF Infra feel confident the existing mirror network will support this? - can SF become a part of that mirror network in a sensible way? - note that SF does not provide direct links to the download, they provide an intermediate page with advertising - should any of the old OOo mirrors be incorporated into the ASF mirror system? - will they want the additional overhead brought by all other ASF projects? My own feeling is that the ASF infra team would not really be interesting in changing the mirror system in any way, however I am *not* member of the infra team and cannot speak for them. Joe, of course is. If the PPMC sees the need to explore SF hosting then I suggest someone picks this up and liaises between ASF Infra, AOO and SF. IF the PPMC is confident that the existing mirror system is sufficient then no need to revisit. Ross
Re: Sourceforge and AOO 3.4 distribution
+1... I trust Joe's PoV here implicitly. On Mar 19, 2012, at 2:54 PM, Joe Schaefer wrote: Given the initial feedback Jurgen provided on the infra lists about the potential number of downloads a day and expected size of each download, I think it would be prudent to take advantage of any assistance sourceforge might be able to provide here. What I'm thinking is some sort of hybrid approach where the recommended default download is a sourceforge link with the Apache mirrors as auxiliary optional links further down the page.
Re: Sourceforge and AOO 3.4 distribution
OK, Roberto is on this list, so let's see what can be worked out. My exploratory discussions with Roberto indicated that SF are willing to work with us on a decent solution. Joe can I assume you are happy to represent ASF infra here, or should we take it straight to the infra lists? Ross Sent from my mobile device, please forgive errors and brevity. On Mar 19, 2012 6:54 PM, Joe Schaefer joe_schae...@yahoo.com wrote: Given the initial feedback Jurgen provided on the infra lists about the potential number of downloads a day and expected size of each download, I think it would be prudent to take advantage of any assistance sourceforge might be able to provide here. What I'm thinking is some sort of hybrid approach where the recommended default download is a sourceforge link with the Apache mirrors as auxiliary optional links further down the page. - Original Message - From: Joe Schaefer joe_schae...@yahoo.com To: ooo-dev@incubator.apache.org ooo-dev@incubator.apache.org Cc: Sent: Monday, March 19, 2012 11:23 AM Subject: Re: Sourceforge and AOO 3.4 distribution - Original Message - From: Ross Gardler rgard...@opendirective.com To: ooo-dev@incubator.apache.org Cc: Sent: Monday, March 19, 2012 11:03 AM Subject: Sourceforge and AOO 3.4 distribution I just had a call with Roberto from SourceForge in which he updated me on what they've done with the templates and extensions sites. I asked Roberto to send a summary to this list, but I just wanted to extend my thanks to him and the team at SourceForge, along with the people here in AOO and ASF infra who have helped. Roberto also asked if there is anything SF can do to help distributing the AOO 3.4 We've discussed this a few times but as we are now close to a release I think it is worth recapping and making sure everything is lined up OK. - what are the likely bandwidth requirements when the release goes out? As far as Infra is concerned, it will depend on the total size of the artifacts being released multiplied by the number of mirrors that need to download it from us over a 6 hour period. We are considering rate-limiting our rsync service to lower the peak bandwidth needed. - does ASF Infra feel confident the existing mirror network will support this? I'd say most mirrors won't object once we give them a heads-up about how much additional disk space and bandwidth will be required. It would help if the PPMC could provide infra with an estimate of the expected number of total downloads per day during the first week or two of release, combined with the typical download size, so we may provide that information to the mirror operators and let them decide whether to stay with us or drop out. - can SF become a part of that mirror network in a sensible way? - note that SF does not provide direct links to the download, they provide an intermediate page with advertising The advertising does not exactly thrill me, and really isn't compatible with how our mirror scripts work. - should any of the old OOo mirrors be incorporated into the ASF mirror system? - will they want the additional overhead brought by all other ASF projects? My own feeling is that the ASF infra team would not really be interesting in changing the mirror system in any way, however I am *not* member of the infra team and cannot speak for them. Joe, of course is. If the PPMC sees the need to explore SF hosting then I suggest someone picks this up and liaises between ASF Infra, AOO and SF. IF the PPMC is confident that the existing mirror system is sufficient then no need to revisit. Ross -- Ross Gardler (@rgardler) Programme Leader (Open Development) OpenDirective http://opendirective.com
Re: Sourceforge and AOO 3.4 distribution
Let's leave the conversation off the infra lists so Roberto can see them all. I'm happy to represent infra here. From: Ross Gardler rgard...@opendirective.com To: ooo-dev@incubator.apache.org; Joe Schaefer joe_schae...@yahoo.com Sent: Monday, March 19, 2012 3:54 PM Subject: Re: Sourceforge and AOO 3.4 distribution OK, Roberto is on this list, so let's see what can be worked out. My exploratory discussions with Roberto indicated that SF are willing to work with us on a decent solution. Joe can I assume you are happy to represent ASF infra here, or should we take it straight to the infra lists? Ross Sent from my mobile device, please forgive errors and brevity. On Mar 19, 2012 6:54 PM, Joe Schaefer joe_schae...@yahoo.com wrote: Given the initial feedback Jurgen provided on the infra lists about the potential number of downloads a day and expected size of each download, I think it would be prudent to take advantage of any assistance sourceforge might be able to provide here. What I'm thinking is some sort of hybrid approach where the recommended default download is a sourceforge link with the Apache mirrors as auxiliary optional links further down the page. - Original Message - From: Joe Schaefer joe_schae...@yahoo.com To: ooo-dev@incubator.apache.org ooo-dev@incubator.apache.org Cc: Sent: Monday, March 19, 2012 11:23 AM Subject: Re: Sourceforge and AOO 3.4 distribution - Original Message - From: Ross Gardler rgard...@opendirective.com To: ooo-dev@incubator.apache.org Cc: Sent: Monday, March 19, 2012 11:03 AM Subject: Sourceforge and AOO 3.4 distribution I just had a call with Roberto from SourceForge in which he updated me on what they've done with the templates and extensions sites. I asked Roberto to send a summary to this list, but I just wanted to extend my thanks to him and the team at SourceForge, along with the people here in AOO and ASF infra who have helped. Roberto also asked if there is anything SF can do to help distributing the AOO 3.4 We've discussed this a few times but as we are now close to a release I think it is worth recapping and making sure everything is lined up OK. - what are the likely bandwidth requirements when the release goes out? As far as Infra is concerned, it will depend on the total size of the artifacts being released multiplied by the number of mirrors that need to download it from us over a 6 hour period. We are considering rate-limiting our rsync service to lower the peak bandwidth needed. - does ASF Infra feel confident the existing mirror network will support this? I'd say most mirrors won't object once we give them a heads-up about how much additional disk space and bandwidth will be required. It would help if the PPMC could provide infra with an estimate of the expected number of total downloads per day during the first week or two of release, combined with the typical download size, so we may provide that information to the mirror operators and let them decide whether to stay with us or drop out. - can SF become a part of that mirror network in a sensible way? - note that SF does not provide direct links to the download, they provide an intermediate page with advertising The advertising does not exactly thrill me, and really isn't compatible with how our mirror scripts work. - should any of the old OOo mirrors be incorporated into the ASF mirror system? - will they want the additional overhead brought by all other ASF projects? My own feeling is that the ASF infra team would not really be interesting in changing the mirror system in any way, however I am *not* member of the infra team and cannot speak for them. Joe, of course is. If the PPMC sees the need to explore SF hosting then I suggest someone picks this up and liaises between ASF Infra, AOO and SF. IF the PPMC is confident that the existing mirror system is sufficient then no need to revisit. Ross -- Ross Gardler (@rgardler) Programme Leader (Open Development) OpenDirective http://opendirective.com
Re: Sourceforge and AOO 3.4 distribution
FWIW the ballpark figures we have today Roberto are roughly 12GB worth of release artifacts and about100TB / day worth of download traffic. From: Joe Schaefer joe_schae...@yahoo.com To: Ross Gardler rgard...@opendirective.com; ooo-dev@incubator.apache.org ooo-dev@incubator.apache.org Sent: Monday, March 19, 2012 3:55 PM Subject: Re: Sourceforge and AOO 3.4 distribution Let's leave the conversation off the infra lists so Roberto can see them all. I'm happy to represent infra here. From: Ross Gardler rgard...@opendirective.com To: ooo-dev@incubator.apache.org; Joe Schaefer joe_schae...@yahoo.com Sent: Monday, March 19, 2012 3:54 PM Subject: Re: Sourceforge and AOO 3.4 distribution OK, Roberto is on this list, so let's see what can be worked out. My exploratory discussions with Roberto indicated that SF are willing to work with us on a decent solution. Joe can I assume you are happy to represent ASF infra here, or should we take it straight to the infra lists? Ross Sent from my mobile device, please forgive errors and brevity. On Mar 19, 2012 6:54 PM, Joe Schaefer joe_schae...@yahoo.com wrote: Given the initial feedback Jurgen provided on the infra lists about the potential number of downloads a day and expected size of each download, I think it would be prudent to take advantage of any assistance sourceforge might be able to provide here. What I'm thinking is some sort of hybrid approach where the recommended default download is a sourceforge link with the Apache mirrors as auxiliary optional links further down the page. - Original Message - From: Joe Schaefer joe_schae...@yahoo.com To: ooo-dev@incubator.apache.org ooo-dev@incubator.apache.org Cc: Sent: Monday, March 19, 2012 11:23 AM Subject: Re: Sourceforge and AOO 3.4 distribution - Original Message - From: Ross Gardler rgard...@opendirective.com To: ooo-dev@incubator.apache.org Cc: Sent: Monday, March 19, 2012 11:03 AM Subject: Sourceforge and AOO 3.4 distribution I just had a call with Roberto from SourceForge in which he updated me on what they've done with the templates and extensions sites. I asked Roberto to send a summary to this list, but I just wanted to extend my thanks to him and the team at SourceForge, along with the people here in AOO and ASF infra who have helped. Roberto also asked if there is anything SF can do to help distributing the AOO 3.4 We've discussed this a few times but as we are now close to a release I think it is worth recapping and making sure everything is lined up OK. - what are the likely bandwidth requirements when the release goes out? As far as Infra is concerned, it will depend on the total size of the artifacts being released multiplied by the number of mirrors that need to download it from us over a 6 hour period. We are considering rate-limiting our rsync service to lower the peak bandwidth needed. - does ASF Infra feel confident the existing mirror network will support this? I'd say most mirrors won't object once we give them a heads-up about how much additional disk space and bandwidth will be required. It would help if the PPMC could provide infra with an estimate of the expected number of total downloads per day during the first week or two of release, combined with the typical download size, so we may provide that information to the mirror operators and let them decide whether to stay with us or drop out. - can SF become a part of that mirror network in a sensible way? - note that SF does not provide direct links to the download, they provide an intermediate page with advertising The advertising does not exactly thrill me, and really isn't compatible with how our mirror scripts work. - should any of the old OOo mirrors be incorporated into the ASF mirror system? - will they want the additional overhead brought by all other ASF projects? My own feeling is that the ASF infra team would not really be interesting in changing the mirror system in any way, however I am *not* member of the infra team and cannot speak for them. Joe, of course is. If the PPMC sees the need to explore SF hosting then I suggest someone picks this up and liaises between ASF Infra, AOO and SF. IF the PPMC is confident that the existing mirror system is sufficient then no need to revisit. Ross -- Ross Gardler (@rgardler) Programme Leader (Open Development) OpenDirective http://opendirective.com
Re: Sourceforge and AOO 3.4 distribution
Ross and Joe, Thank you both for your attention to this today. I am looking forward to Roberto's responses. Regards, Dave On Mar 19, 2012, at 12:55 PM, Joe Schaefer wrote: Let's leave the conversation off the infra lists so Roberto can see them all. I'm happy to represent infra here. From: Ross Gardler rgard...@opendirective.com To: ooo-dev@incubator.apache.org; Joe Schaefer joe_schae...@yahoo.com Sent: Monday, March 19, 2012 3:54 PM Subject: Re: Sourceforge and AOO 3.4 distribution OK, Roberto is on this list, so let's see what can be worked out. My exploratory discussions with Roberto indicated that SF are willing to work with us on a decent solution. Joe can I assume you are happy to represent ASF infra here, or should we take it straight to the infra lists? Ross Sent from my mobile device, please forgive errors and brevity. On Mar 19, 2012 6:54 PM, Joe Schaefer joe_schae...@yahoo.com wrote: Given the initial feedback Jurgen provided on the infra lists about the potential number of downloads a day and expected size of each download, I think it would be prudent to take advantage of any assistance sourceforge might be able to provide here. What I'm thinking is some sort of hybrid approach where the recommended default download is a sourceforge link with the Apache mirrors as auxiliary optional links further down the page. - Original Message - From: Joe Schaefer joe_schae...@yahoo.com To: ooo-dev@incubator.apache.org ooo-dev@incubator.apache.org Cc: Sent: Monday, March 19, 2012 11:23 AM Subject: Re: Sourceforge and AOO 3.4 distribution - Original Message - From: Ross Gardler rgard...@opendirective.com To: ooo-dev@incubator.apache.org Cc: Sent: Monday, March 19, 2012 11:03 AM Subject: Sourceforge and AOO 3.4 distribution I just had a call with Roberto from SourceForge in which he updated me on what they've done with the templates and extensions sites. I asked Roberto to send a summary to this list, but I just wanted to extend my thanks to him and the team at SourceForge, along with the people here in AOO and ASF infra who have helped. Roberto also asked if there is anything SF can do to help distributing the AOO 3.4 We've discussed this a few times but as we are now close to a release I think it is worth recapping and making sure everything is lined up OK. - what are the likely bandwidth requirements when the release goes out? As far as Infra is concerned, it will depend on the total size of the artifacts being released multiplied by the number of mirrors that need to download it from us over a 6 hour period. We are considering rate-limiting our rsync service to lower the peak bandwidth needed. - does ASF Infra feel confident the existing mirror network will support this? I'd say most mirrors won't object once we give them a heads-up about how much additional disk space and bandwidth will be required. It would help if the PPMC could provide infra with an estimate of the expected number of total downloads per day during the first week or two of release, combined with the typical download size, so we may provide that information to the mirror operators and let them decide whether to stay with us or drop out. - can SF become a part of that mirror network in a sensible way? - note that SF does not provide direct links to the download, they provide an intermediate page with advertising The advertising does not exactly thrill me, and really isn't compatible with how our mirror scripts work. - should any of the old OOo mirrors be incorporated into the ASF mirror system? - will they want the additional overhead brought by all other ASF projects? My own feeling is that the ASF infra team would not really be interesting in changing the mirror system in any way, however I am *not* member of the infra team and cannot speak for them. Joe, of course is. If the PPMC sees the need to explore SF hosting then I suggest someone picks this up and liaises between ASF Infra, AOO and SF. IF the PPMC is confident that the existing mirror system is sufficient then no need to revisit. Ross -- Ross Gardler (@rgardler) Programme Leader (Open Development) OpenDirective http://opendirective.com
