Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Am 10/28/2011 03:38 AM, schrieb Gavin McDonald: -Original Message- From: Marcus (OOo) [mailto:marcus.m...@wtnet.de] Sent: Friday, 28 October 2011 9:13 AM To: ooo-dev@incubator.apache.org Cc: Peter Pöml Subject: Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance Am 10/27/2011 11:36 AM, schrieb Gavin McDonald: -Original Message- From: Ross Gardler [mailto:rgard...@opendirective.com] Sent: Thursday, 27 October 2011 10:48 AM To: ooo-dev@incubator.apache.org Cc: Peter Pöml Subject: Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance Sent from my mobile device, please forgive errors and brevity. On Oct 27, 2011 1:22 AM, Marcus (OOo)marcus.m...@wtnet.de wrote: Am 10/27/2011 02:03 AM, schrieb Ross Gardler: Sent from my mobile device, please forgive errors and brevity. On Oct 27, 2011 12:37 AM, Marcus (OOo)marcus.m...@wtnet.de wrote: Am 10/26/2011 11:57 PM, schrieb Peter Pöml: Setting up MirrorBrain would be one way, but it would require replication additional configuration (for instance, download statistics) that we have built on the current download server (download.services.openoffice.org). Another way would be to simply have a virtual machine, where we move the current server to. That would cause the least effort, I guess. +1 this should be really our goal for now ... ... Starting from scratch would mean to lose a lot of the previos work -- and I really mean lots, which I dare to judge because I spent a lot of time with download.services.openoffice.org. On the other hand, having MirrorBrain at the core of the ASF's mirror system could be interesting for other projects, too. I know closer.cgi but I'm sure that MirrorBrain could serve the ASF well. (Well possible as an addition, rather than a replacement, for a soft transition.) That might outweigh the pain of creating OOo's download service from scratch in a different environment. ... and this the long term goal. The ASF can really benefit from this way of downloading software. Currently you have to choose a mirror, then change to the directory structure where the respective binary is located and finally download it. This is not correct. I have no idea if the ASF can benefit from MirrorBrain or not, but if your justification for such a statement is based on the above erroneous analysis of the current mirror system then I have my concerns. I OK, thats what I've done to come to my point: - browse to http://www.apache.org/; - click on Download top right - choose a mirror - change to the dir structure to your file - now download Is there an easier way to get software? E.g. http://httpd.apache.org/download.cgi#apache22 It auto selects the nearest mirror and provides appropriate download links - a single click. Want it pretty with big blue buttons? It's just HTML This was discussed early in the AOOo podlings existence.see archives for.more. I would like us to move on from this Topic, I have said before and I will say it again, ASF Infra will NOT support mirrorbrain. We have one mirror system and it deals with 200 or so projects. OpenOffice project will use the current ASF supported mirror system. End of story. If you see this like black and white then I can do this, too: If we don't get the support that AOO needs then we can outsource the download easily. EOD (yes, this wasn't meant seriously) It's sad that you brush it simply aside without to see the chance to at least try to improve things. Excuse me? I am doing quite a lot already thank you. You do not our infrastructure or how it works, I do. And we are all very thankful for this. This wasn't meant as offense. Sorry if you have understood it this way. Why OOo is special: - OOo 3.3 consists of ~1.000 files with ~70 GB Correct. - in the peak with 300,000 downloads per day That’s fine. The mirrors will handle that. OK, forget the other numbers as of course only this is important for the topic. Even when we won't get back the download numbers in the middle run due to the long silence of the last months and reduce the number by lets say 50% we would be still by far the largest ASF project. Doesn’t give you the right to try and bully me or the rest of the infra team into giving you want you want. In fact, it will have the opposite effect. Stop. I haven't bully you and never the entire infra team. Again, sorry if you have understood it this way. But you are right ... That said, let us get these other migrations done and we can see how/if the previous releases can be integrated. ... currently we have indeed other more important road works on the plan. And we are working on them. The downloads will be sorted don’t worry, but if I can help it, it will be using the same system as all the other projects. If I am wrong about that, I will fess up if it happens. Marcus
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Am 10/28/2011 10:02 AM, schrieb Daniel Shahaf: Marcus (OOo) wrote on Fri, Oct 28, 2011 at 01:12:33 +0200: Why OOo is special: - OOo 3.3 consists of ~1.000 files with ~70 GB - in the peak with 300,000 downloads per day That's the relevant part of your email. I agree that 70GB per release Yes, forget the other numbers as only the download part is of course the topic-related part. is quite an outlier (and will cause issues not only related to mirroring). When the PPMC wants to ask infra something or wants infra to do something, email us on the usual channels. OK Marcus
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Am 10/27/2011 11:36 AM, schrieb Gavin McDonald: -Original Message- From: Ross Gardler [mailto:rgard...@opendirective.com] Sent: Thursday, 27 October 2011 10:48 AM To: ooo-dev@incubator.apache.org Cc: Peter Pöml Subject: Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance Sent from my mobile device, please forgive errors and brevity. On Oct 27, 2011 1:22 AM, Marcus (OOo)marcus.m...@wtnet.de wrote: Am 10/27/2011 02:03 AM, schrieb Ross Gardler: Sent from my mobile device, please forgive errors and brevity. On Oct 27, 2011 12:37 AM, Marcus (OOo)marcus.m...@wtnet.de wrote: Am 10/26/2011 11:57 PM, schrieb Peter Pöml: Setting up MirrorBrain would be one way, but it would require replication additional configuration (for instance, download statistics) that we have built on the current download server (download.services.openoffice.org). Another way would be to simply have a virtual machine, where we move the current server to. That would cause the least effort, I guess. +1 this should be really our goal for now ... ... Starting from scratch would mean to lose a lot of the previos work -- and I really mean lots, which I dare to judge because I spent a lot of time with download.services.openoffice.org. On the other hand, having MirrorBrain at the core of the ASF's mirror system could be interesting for other projects, too. I know closer.cgi but I'm sure that MirrorBrain could serve the ASF well. (Well possible as an addition, rather than a replacement, for a soft transition.) That might outweigh the pain of creating OOo's download service from scratch in a different environment. ... and this the long term goal. The ASF can really benefit from this way of downloading software. Currently you have to choose a mirror, then change to the directory structure where the respective binary is located and finally download it. This is not correct. I have no idea if the ASF can benefit from MirrorBrain or not, but if your justification for such a statement is based on the above erroneous analysis of the current mirror system then I have my concerns. I OK, thats what I've done to come to my point: - browse to http://www.apache.org/; - click on Download top right - choose a mirror - change to the dir structure to your file - now download Is there an easier way to get software? E.g. http://httpd.apache.org/download.cgi#apache22 It auto selects the nearest mirror and provides appropriate download links - a single click. Want it pretty with big blue buttons? It's just HTML This was discussed early in the AOOo podlings existence.see archives for.more. I would like us to move on from this Topic, I have said before and I will say it again, ASF Infra will NOT support mirrorbrain. We have one mirror system and it deals with 200 or so projects. OpenOffice project will use the current ASF supported mirror system. End of story. If you see this like black and white then I can do this, too: If we don't get the support that AOO needs then we can outsource the download easily. EOD (yes, this wasn't meant seriously) It's sad that you brush it simply aside without to see the chance to at least try to improve things. Why OOo is special: - over 70 committers (more than 10x of other projects, right ?) - several millons of code lines - over 70 supported languages - 4 supported CPU architectures - 4 supported operating systems - OOo 3.3 consists of ~1.000 files with ~70 GB - in the peak with 300,000 downloads per day With these short facts IMHO everybody should be able to see that this is not just another project and that it needs special treatment when needed. Even when we won't get back the download numbers in the middle run due to the long silence of the last months and reduce the number by lets say 50% we would be still by far the largest ASF project. But you are right ... That said, let us get these other migrations done and we can see how/if the previous releases can be integrated. ... currently we have indeed other more important road works on the plan. Marcus
RE: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
-Original Message- From: Marcus (OOo) [mailto:marcus.m...@wtnet.de] Sent: Friday, 28 October 2011 9:13 AM To: ooo-dev@incubator.apache.org Cc: Peter Pöml Subject: Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance Am 10/27/2011 11:36 AM, schrieb Gavin McDonald: -Original Message- From: Ross Gardler [mailto:rgard...@opendirective.com] Sent: Thursday, 27 October 2011 10:48 AM To: ooo-dev@incubator.apache.org Cc: Peter Pöml Subject: Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance Sent from my mobile device, please forgive errors and brevity. On Oct 27, 2011 1:22 AM, Marcus (OOo)marcus.m...@wtnet.de wrote: Am 10/27/2011 02:03 AM, schrieb Ross Gardler: Sent from my mobile device, please forgive errors and brevity. On Oct 27, 2011 12:37 AM, Marcus (OOo)marcus.m...@wtnet.de wrote: Am 10/26/2011 11:57 PM, schrieb Peter Pöml: Setting up MirrorBrain would be one way, but it would require replication additional configuration (for instance, download statistics) that we have built on the current download server (download.services.openoffice.org). Another way would be to simply have a virtual machine, where we move the current server to. That would cause the least effort, I guess. +1 this should be really our goal for now ... ... Starting from scratch would mean to lose a lot of the previos work -- and I really mean lots, which I dare to judge because I spent a lot of time with download.services.openoffice.org. On the other hand, having MirrorBrain at the core of the ASF's mirror system could be interesting for other projects, too. I know closer.cgi but I'm sure that MirrorBrain could serve the ASF well. (Well possible as an addition, rather than a replacement, for a soft transition.) That might outweigh the pain of creating OOo's download service from scratch in a different environment. ... and this the long term goal. The ASF can really benefit from this way of downloading software. Currently you have to choose a mirror, then change to the directory structure where the respective binary is located and finally download it. This is not correct. I have no idea if the ASF can benefit from MirrorBrain or not, but if your justification for such a statement is based on the above erroneous analysis of the current mirror system then I have my concerns. I OK, thats what I've done to come to my point: - browse to http://www.apache.org/; - click on Download top right - choose a mirror - change to the dir structure to your file - now download Is there an easier way to get software? E.g. http://httpd.apache.org/download.cgi#apache22 It auto selects the nearest mirror and provides appropriate download links - a single click. Want it pretty with big blue buttons? It's just HTML This was discussed early in the AOOo podlings existence.see archives for.more. I would like us to move on from this Topic, I have said before and I will say it again, ASF Infra will NOT support mirrorbrain. We have one mirror system and it deals with 200 or so projects. OpenOffice project will use the current ASF supported mirror system. End of story. If you see this like black and white then I can do this, too: If we don't get the support that AOO needs then we can outsource the download easily. EOD (yes, this wasn't meant seriously) It's sad that you brush it simply aside without to see the chance to at least try to improve things. Excuse me? I am doing quite a lot already thank you. You do not our infrastructure or how it works, I do. Why OOo is special: - over 70 committers (more than 10x of other projects, right ?) Wrong Subversion 66 hadoop 64 commons 103 lucene 61 cocoon 81 Geronimo 64 Shall I go on, 10x other projects where on earth did you pull that gem from? - several millons of code lines LOC does not make a project special. Perhaps as things improve with efficiency those will come down. - over 70 supported languages Other project support other languages too. - 4 supported CPU architectures Some projects support more than that, your point is? - 4 supported operating systems Some projects support more than that, your point is? - OOo 3.3 consists of ~1.000 files with ~70 GB Correct. - in the peak with 300,000 downloads per day Thats fine. The mirrors will handle that. With these short facts IMHO everybody should be able to see that this is not just another project and that it needs special treatment when needed. sorry, you already are getting special treatment in other ways, but if every Tom Dick and Sally from this project got exactly what they asked for, they would also be paying for more servers, more bandwidth and more people to look after it all. Does everything
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
On Oct 26, 2011, at 12:11 PM, Marcus (OOo) wrote: Am 10/24/2011 03:08 AM, schrieb Marcus (OOo): Am 10/23/2011 11:50 PM, schrieb Peter Pöml: On Sat, Oct 22, 2011 at 01:15:34 +0200, Marcus (OOo) wrote: Hi Peter, it seems the end is near and the download.services.openoffice.org host on Oracle side with our Mirrorbrain instance will be shutdown in one week. Okay... Little question, wouldn't it make sense, at this point, to take the download.s.o.o VM and move it somewhere else? I don't have the disk space myself, but maybe somebody else has? Perhaps the ASF can provide the space? (Other than disk space, the requirements are pretty modest.) Then the entire download service could continue to run, with very low effort, and without the need to build everything from scratch for now. The problem is that the ASF do not want to host and provide services of special software for single projects. I can understand this as even the ASF infra is a team of volunteers and their time is limited as it is for all others. They are building a lot special for AOOo. THANKS! So, it depends. If we define a stack for services and get support then it is a lot like the MWiki and Forums. Infrastructure will help secure the servers, but the project will need to provide sysadmins. I've heard that Jails are available. Furthermore, I don't know details about the VM setup and where it's detailed located. Do you? However, I like your idea. It requires indeed only a bit diskspace and internet access. The maintainance could be done by us, the project members. If you would share a bit of your knowledge then I could takeover the admin role. ;-) @Peter: Can you tell me how to get to the VM? Or at least which important config files to save? Are there requirements for more than what is on the mirrorbrain page? http://mirrorbrain.org/requirements/ Extracting the list gives: Apache HTTPD server mod_mirrorbrain mod_form mod_geoip / libGeoIP PostgreSQL (or Apache DBD API compatible driver) mod_asn ? • Python and some modules: psycopg2, sqlobject, cmdln. • Perl and some modules: Config::IniFiles, libwww::perl, DBD::Pg, Digest::MD4, Date::Parse If we are on ASF hardware then we won't want to serve a local copy: So, Alternatively, a pseudo file tree can created locally. In that case, MirrorBrain needs to be configured to never deliver files directly. http://mirrorbrain.org/archive/mirrorbrain/0045.html @List: Has anybody an idea about where to host this service? It doesn't need to be necessarily inside the ASF. Really nobody? If the mirrorbrain instructions are that simple and can be easily supplemented with detail. Like what database is currently being used on download.s.oo.o? and which version? As we cannot buildup a solution that is running, tested and long-term-proven in a few days I would like to ask you if we can switch to your openoffice.mirrorbrain.org instance as long as we have no other solution at hand. In the past it was a very reliable host that we have used when there were outages (mostly unplanned as you know) and doesn't resulted in a significant higher load on the host. So, it would help us very much to have a stable download section until we have an own solution here at ASF. Yes, please use my host for now! Great, thanks a lot. Thanks in advance and have a nice, sunny weekend. Thanks a lot. Same to you all, Marcus Regards, Dave
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Hi, On Wed, Oct 26, 2011 at 01:43:20 -0700, Dave Fisher wrote: Are there requirements for more than what is on the mirrorbrain page? The page should list all essential requirements. http://mirrorbrain.org/requirements/ Extracting the list gives: Apache HTTPD server mod_mirrorbrain mod_form mod_geoip / libGeoIP PostgreSQL (or Apache DBD API compatible driver) While the DBD framework would allow to use other databases, MirrorBrain depends on features that only PostgreSQL offers. So this is a must, I'm sorry ;-) mod_asn ? Highly useful when there are many mirrors, and when downloads are large. Both is the case with OOo. Thus, we have used mod_asn for OOo downloads so far and I would really recommend to continue using it. If we are on ASF hardware then we won't want to serve a local copy: So, Alternatively, a pseudo file tree can created locally. In that case, MirrorBrain needs to be configured to never deliver files directly. http://mirrorbrain.org/archive/mirrorbrain/0045.html MirrorBrain is more than a file server or redirector. It automates the creation of checksums, hashes, torrents, metalinks and serves cryptographic signatures. Files need to be present locally to exploit these features. These features are useful for OOo because downloads are generally large, which requires reliable transfer. @List: Has anybody an idea about where to host this service? It doesn't need to be necessarily inside the ASF. Really nobody? If the mirrorbrain instructions are that simple and can be easily supplemented with detail. Setting up MirrorBrain would be one way, but it would require replication additional configuration (for instance, download statistics) that we have built on the current download server (download.services.openoffice.org). Another way would be to simply have a virtual machine, where we move the current server to. That would cause the least effort, I guess. Starting from scratch would mean to lose a lot of the previos work -- and I really mean lots, which I dare to judge because I spent a lot of time with download.services.openoffice.org. On the other hand, having MirrorBrain at the core of the ASF's mirror system could be interesting for other projects, too. I know closer.cgi but I'm sure that MirrorBrain could serve the ASF well. (Well possible as an addition, rather than a replacement, for a soft transition.) That might outweigh the pain of creating OOo's download service from scratch in a different environment. Are you involved in maintaining ASF infrastructure, or know people who are? Like what database is currently being used on download.s.oo.o? and which version? Uhm, PostgreSQL 8.4.4. (But any 8.3/8.4 version should do.) Thanks, Peter pgpLYGG9McsVD.pgp Description: PGP signature
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
On Wed, Oct 26, 2011 at 5:57 PM, Peter Pöml pe...@poeml.de wrote: The page should list all essential requirements. http://mirrorbrain.org/requirements/ Does MirrorBrain allow downloaders to specify the mirror they want to use? I didn't see that as a feature. This is necessary for people operating behind firewalls that block websites based on type. For example, my company's firewall doesn't allow access to many filesharing sites and certain types of businesses, but does allow access to universities. This means when I get an update for an Apache product I often need to manually select a university mirror. Don
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Am 10/27/2011 12:43 AM, schrieb Donald Whytock: On Wed, Oct 26, 2011 at 5:57 PM, Peter Pömlpe...@poeml.de wrote: The page should list all essential requirements. http://mirrorbrain.org/requirements/ Does MirrorBrain allow downloaders to specify the mirror they want to use? I didn't see that as a feature. This is necessary for people operating behind firewalls that block websites based on type. For example, my company's firewall doesn't allow access to many filesharing sites and certain types of businesses, but does allow access to universities. This means when I get an update for an Apache product I often need to manually select a university mirror. I would say not directly but it's possible: Example: If you want to download the OOo 3.4 Beta you normally would download from here for your favorite OS and language: http://download.openoffice.org/all_beta.html When the automatically chosen mirror is not suitable, then you can use the Details webpage that MirrorBrain creates for every file to look for other mirrors. E.g.: http://download.services.openoffice.org/files/extended/3.4beta/ http://download.services.openoffice.org/files/extended/3.4beta/OOo-Dev_3.4beta_20110411_Linux_x86-64_install-rpm_en-US.tar.gz.mirrorlist It's not that obvious but IMHO simple enough. But I guess even this could be more simplified. ;-) Marcus
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Am 10/26/2011 11:57 PM, schrieb Peter Pöml: Hi, On Wed, Oct 26, 2011 at 01:43:20 -0700, Dave Fisher wrote: Are there requirements for more than what is on the mirrorbrain page? The page should list all essential requirements. http://mirrorbrain.org/requirements/ Extracting the list gives: Apache HTTPD server mod_mirrorbrain mod_form mod_geoip / libGeoIP PostgreSQL (or Apache DBD API compatible driver) While the DBD framework would allow to use other databases, MirrorBrain depends on features that only PostgreSQL offers. So this is a must, I'm sorry ;-) mod_asn ? Highly useful when there are many mirrors, and when downloads are large. Both is the case with OOo. Thus, we have used mod_asn for OOo downloads so far and I would really recommend to continue using it. If we are on ASF hardware then we won't want to serve a local copy: So, Alternatively, a pseudo file tree can created locally. In that case, MirrorBrain needs to be configured to never deliver files directly. http://mirrorbrain.org/archive/mirrorbrain/0045.html MirrorBrain is more than a file server or redirector. It automates the creation of checksums, hashes, torrents, metalinks and serves cryptographic signatures. Files need to be present locally to exploit these features. These features are useful for OOo because downloads are generally large, which requires reliable transfer. @List: Has anybody an idea about where to host this service? It doesn't need to be necessarily inside the ASF. Really nobody? If the mirrorbrain instructions are that simple and can be easily supplemented with detail. Setting up MirrorBrain would be one way, but it would require replication additional configuration (for instance, download statistics) that we have built on the current download server (download.services.openoffice.org). Another way would be to simply have a virtual machine, where we move the current server to. That would cause the least effort, I guess. +1 this should be really our goal for now ... Starting from scratch would mean to lose a lot of the previos work -- and I really mean lots, which I dare to judge because I spent a lot of time with download.services.openoffice.org. On the other hand, having MirrorBrain at the core of the ASF's mirror system could be interesting for other projects, too. I know closer.cgi but I'm sure that MirrorBrain could serve the ASF well. (Well possible as an addition, rather than a replacement, for a soft transition.) That might outweigh the pain of creating OOo's download service from scratch in a different environment. ... and this the long term goal. The ASF can really benefit from this way of downloading software. Currently you have to choose a mirror, then change to the directory structure where the respective binary is located and finally download it. OK, the target audience the ASF currently has is developer-oriented, so this is not too difficult for them. ;-) However, it could be indeed easier. But I try to think more as an enduser. And these people need to have a simple solution. That means a link on a webpage where they click on and the download of the file is starting. IMHO we have already a very simple solution you can see on http://download.openoffice.org; and I would be happy to help to buildup a similar thing for the entire ASF. @Donald: Of course we can keep the current download behavior to satisfy also users that needs to download from a specific mirror server. Marcus Are you involved in maintaining ASF infrastructure, or know people who are? Like what database is currently being used on download.s.oo.o? and which version? Uhm, PostgreSQL 8.4.4. (But any 8.3/8.4 version should do.) Thanks, Peter
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Sent from my mobile device, please forgive errors and brevity. On Oct 27, 2011 1:22 AM, Marcus (OOo) marcus.m...@wtnet.de wrote: Am 10/27/2011 02:03 AM, schrieb Ross Gardler: Sent from my mobile device, please forgive errors and brevity. On Oct 27, 2011 12:37 AM, Marcus (OOo)marcus.m...@wtnet.de wrote: Am 10/26/2011 11:57 PM, schrieb Peter Pöml: Setting up MirrorBrain would be one way, but it would require replication additional configuration (for instance, download statistics) that we have built on the current download server (download.services.openoffice.org). Another way would be to simply have a virtual machine, where we move the current server to. That would cause the least effort, I guess. +1 this should be really our goal for now ... ... Starting from scratch would mean to lose a lot of the previos work -- and I really mean lots, which I dare to judge because I spent a lot of time with download.services.openoffice.org. On the other hand, having MirrorBrain at the core of the ASF's mirror system could be interesting for other projects, too. I know closer.cgi but I'm sure that MirrorBrain could serve the ASF well. (Well possible as an addition, rather than a replacement, for a soft transition.) That might outweigh the pain of creating OOo's download service from scratch in a different environment. ... and this the long term goal. The ASF can really benefit from this way of downloading software. Currently you have to choose a mirror, then change to the directory structure where the respective binary is located and finally download it. This is not correct. I have no idea if the ASF can benefit from MirrorBrain or not, but if your justification for such a statement is based on the above erroneous analysis of the current mirror system then I have my concerns. I OK, thats what I've done to come to my point: - browse to http://www.apache.org/; - click on Download top right - choose a mirror - change to the dir structure to your file - now download Is there an easier way to get software? E.g. http://httpd.apache.org/download.cgi#apache22 It auto selects the nearest mirror and provides appropriate download links - a single click. Want it pretty with big blue buttons? It's just HTML This was discussed early in the AOOo podlings existence.see archives for.more.
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Hi Dennis, *, On Tue, Oct 25, 2011 at 2:04 AM, Dennis E. Hamilton dennis.hamil...@acm.org wrote: I read somewhere, and I don't know where, that ASF did not want torrents to be used. I'm guessing that the issue is related to ensuring the integrity and authenticity of packaged releases. That doesn't make sense - integrity is assured by bittorrent by providing sha1sums for each chunk. And authenticity can be assured just like it is with regular releases - just include a corresponding signature file within the torrent. I may have dreamed it or I am mixing this up with something else. If those were the only reasons, then they were made-up arguments. But bittorrent only makes sense for larger files anyway. ciao Christian
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
On Mon, Oct 24, 2011 at 2:08 AM, Marcus (OOo) marcus.m...@wtnet.de wrote: snip The problem is that the ASF do not want to host and provide services of special software for single projects. I can understand this as even the ASF infra is a team of volunteers and their time is limited as it is for all others. I think this is a little open to misinterpretation. Hopefully a Mentor will jump in but (until they do) I'll do my best to explain a little bit more about the way infrastructure works here at Apache... The infrastructure team at Apache is an independent, volunteer-led self-organising community of experts. Apache delegates infrastructure to this community, and provides resources to sustain their work[1]. When asking infrastructure for help, it's essential to remember this and engage with them as peers with special expertise. Anyone arriving with a solution or a request for a new service must expect to be challenged to defend and refine their choice of solution. To move back to the particular, this is a migration issue. A valuable service is about to be closed and needs to be migrated. Whether this is right long term solution is open to debate but accepting a service for a temporary period doesn't raise the issues that committing to provide a similar service for all projects forever would. Please explain the problem to infrastructure and ask for their help to find a solution. Robert [1] The team has a budget and some flexibility to bring additional resources - included hired help - when needed. Apache has adequate financial resources but is culturally resistant to committing to additional spending without good reason. Apache values independence. Dependency on funding risks that independence.
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
The issue with bittorrent is that it has become nearly illegal in some countries. I heard about someone being visited by the police in Italy. I do think it is an option but alternate means must be provided. Pedro.
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
On Tue, 25 Oct 2011 05:04:08 -0700 (PDT) Pedro Giffuni p...@apache.org wrote: The issue with bittorrent is that it has become nearly illegal in some countries. I heard about someone being visited by the police in Italy. I do think it is an option but alternate means must be provided. On the Forum we frequently advise users who have a problem downloading to use a torrent; this seems to cure recalcitrant download problems, such as slow (modem) or noisy connections. -- Rory O'Farrell ofarr...@iol.ie
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
On Tue, Oct 25, 2011 at 12:36 PM, Christian Lohmaier cl...@openoffice.org wrote: Hi Dennis, *, On Tue, Oct 25, 2011 at 2:04 AM, Dennis E. Hamilton dennis.hamil...@acm.org wrote: I read somewhere, and I don't know where, that ASF did not want torrents to be used. I'm guessing that the issue is related to ensuring the integrity and authenticity of packaged releases. That doesn't make sense - integrity is assured by bittorrent by providing sha1sums for each chunk. And authenticity can be assured just like it is with regular releases - just include a corresponding signature file within the torrent. Better to download the signature over HTTPS but yes, I see no reason why this approach could not be made to work I may have dreamed it or I am mixing this up with something else. If those were the only reasons, then they were made-up arguments. When engaging with Infrastructure, expect to be challenged and to have to defend any proposal. These lists are open, so expect a range of cluefulness from contributors. The best way to impress the core infrastructure team is for plenty of clueful people from a project to show up and defend the proposal with well research arguments. Giving up and going away is the surest way to lose the argument... Robert
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
On Tue, Oct 25, 2011 at 1:04 AM, Dennis E. Hamilton dennis.hamil...@acm.org wrote: I read somewhere, and I don't know where, that ASF did not want torrents to be used. The meaning and force of this statement is hard to judge without a full context Apache has surprisingly and confusingly little policy, and most of that should be written down Apache encourages wide participation on open lists. Conventionally, opinions expressed on lists are just personal opinions - unless backed by evidence or clear marking[1]. So this is just my personal opinion ;-) Robert [1] wearing a hat :-) http://www.apache.org/foundation/how-it-works.html
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Il 25/10/2011 14.04, Pedro Giffuni ha scritto: The issue with bittorrent is that it has become nearly illegal in some countries. I heard about someone being visited by the police in Italy. There is always somebody visited by our Polizia Postale. ;-) Indeed, the software is legal, of course. It's the way a user utilizes it that may be illegal. BTW, I've seeded LibreOffice for a couple of months and Torrent is a useful tool that helps to distribute among users the bandwidth needed for large downloads. REgards, Gianluca -- Lettura gratuita o acquisto di libri e racconti di fantascienza, fantasy, horror, noir, narrativa fantastica e tradizionale: http://www.letturefantastiche.com/
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Hi Robert, *, On Tue, Oct 25, 2011 at 2:15 PM, Robert Burrell Donkin robertburrelldon...@gmail.com wrote: On Tue, Oct 25, 2011 at 12:36 PM, Christian Lohmaier cl...@openoffice.org wrote: [...] That doesn't make sense - integrity is assured by bittorrent by providing sha1sums for each chunk. And authenticity can be assured just like it is with regular releases - just include a corresponding signature file within the torrent. Better to download the signature over HTTPS but yes, I see no reason why this approach could not be made to work With signature I meant a real signature (gpg signature), not a md5sum or sha1sum file. When it is a cryptographic signature, it doesn't matter how you download it, as it cannot be faked. (of course the user has to get the proper key, but that's a different issue) I may have dreamed it or I am mixing this up with something else. If those were the only reasons, then they were made-up arguments. When engaging with Infrastructure, expect to be challenged and to have to defend any proposal. These lists are open, so expect a range of cluefulness from contributors. The best way to impress the core infrastructure team is for plenty of clueful people from a project to show up and defend the proposal with well research arguments. Giving up and going away is the surest way to lose the argument... With OOo the tracker network[1] was run independently anyway and not hosted on the Oracle or OSUOSL hosted infrastructure. The main tracker was Mike's at utwente, and that mirror also was the initial/main seed for all the releases. There were other trackers linked together via a tracker-hub (backup tracker as well as the hub were provided by Harold). So it is not a matter of infrastructure, but a matter of policy. There's no need for the mechanism to change in my opinion. (torrents are generated automatically as soon as they hit the mirror). So if apache wants to setup their own bt network, they need one capable machine (in terms of bandwidth) server to be the initial seed, and one with almost no resources (can be the same machine of course) to act as tracker. [1] The trackers are *linked*, not separate, all trackers know about every peer, so there is no swarm fragmentation, and you got the fallback in case on of the trackers is down (TDF only uses one single tracker, but webseeds (traditional http/ftp URLs) are included, so even when the tracker is down, the clients can still use regular mirrors and DHT.) ciao Christian
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
On Tue, Oct 25, 2011 at 1:38 PM, Christian Lohmaier cl...@openoffice.org wrote: Hi Robert, *, On Tue, Oct 25, 2011 at 2:15 PM, Robert Burrell Donkin robertburrelldon...@gmail.com wrote: On Tue, Oct 25, 2011 at 12:36 PM, Christian Lohmaier cl...@openoffice.org wrote: [...] That doesn't make sense - integrity is assured by bittorrent by providing sha1sums for each chunk. And authenticity can be assured just like it is with regular releases - just include a corresponding signature file within the torrent. Better to download the signature over HTTPS but yes, I see no reason why this approach could not be made to work With signature I meant a real signature (gpg signature), not a md5sum or sha1sum file. When it is a cryptographic signature, it doesn't matter how you download it, as it cannot be faked. (of course the user has to get the proper key, but that's a different issue) FWIW it's a defense in depth measure[1] I may have dreamed it or I am mixing this up with something else. If those were the only reasons, then they were made-up arguments. When engaging with Infrastructure, expect to be challenged and to have to defend any proposal. These lists are open, so expect a range of cluefulness from contributors. The best way to impress the core infrastructure team is for plenty of clueful people from a project to show up and defend the proposal with well research arguments. Giving up and going away is the surest way to lose the argument... With OOo the tracker network[1] was run independently anyway and not hosted on the Oracle or OSUOSL hosted infrastructure. The main tracker was Mike's at utwente, and that mirror also was the initial/main seed for all the releases. There were other trackers linked together via a tracker-hub (backup tracker as well as the hub were provided by Harold). So it is not a matter of infrastructure, but a matter of policy. Where's the URL for this policy? Robert [1] Consider an attacker with some ability to fabricate convincing signatures. Downloading the signature from a trusted server means that such an attacker would need to replace an existing signature on secure hardware without detection. The small increase in traffic is a small price to pay for this additional defense in depth.
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Am 10/25/2011 02:02 PM, schrieb Robert Burrell Donkin: On Mon, Oct 24, 2011 at 2:08 AM, Marcus (OOo)marcus.m...@wtnet.de wrote: snip The problem is that the ASF do not want to host and provide services of special software for single projects. I can understand this as even the ASF infra is a team of volunteers and their time is limited as it is for all others. I think this is a little open to misinterpretation. Hopefully a Mentor will jump in but (until they do) I'll do my best to explain a little bit more about the way infrastructure works here at Apache... The infrastructure team at Apache is an independent, volunteer-led self-organising community of experts. Apache delegates infrastructure to this community, and provides resources to sustain their work[1]. When asking infrastructure for help, it's essential to remember this and engage with them as peers with special expertise. Anyone arriving with a solution or a request for a new service must expect to be challenged to defend and refine their choice of solution. Thanks for the explaination. I thought it would be a kind of just another project. But it's not. To move back to the particular, this is a migration issue. A valuable service is about to be closed and needs to be migrated. Whether this is right long term solution is open to debate but accepting a service for a temporary period doesn't raise the issues that committing to provide a similar service for all projects forever would. Please explain the problem to infrastructure and ask for their help to find a solution. I know a service like MirrorBrain could be a high value of any Apache project that provides file downloads, so it should be helpful for every project. As it is able keep the overview of all possible mirrors that offers Apache aoftware there needs to be only a single instance and not per project. Let's see what to get until Friday. Otherwise he have to start again. Marcus [1] The team has a budget and some flexibility to bring additional resources - included hired help - when needed. Apache has adequate financial resources but is culturally resistant to committing to additional spending without good reason. Apache values independence. Dependency on funding risks that independence.
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Hi Robert, *, On Tue, Oct 25, 2011 at 5:05 PM, Robert Burrell Donkin robertburrelldon...@gmail.com wrote: On Tue, Oct 25, 2011 at 1:38 PM, Christian Lohmaier cl...@openoffice.org wrote: On Tue, Oct 25, 2011 at 2:15 PM, Robert Burrell Donkin robertburrelldon...@gmail.com wrote: On Tue, Oct 25, 2011 at 12:36 PM, Christian Lohmaier cl...@openoffice.org wrote: [...] Better to download the signature over HTTPS but yes, I see no reason why this approach could not be made to work With signature I meant a real signature (gpg signature), not a md5sum or sha1sum file. When it is a cryptographic signature, it doesn't matter how you download it, as it cannot be faked. (of course the user has to get the proper key, but that's a different issue) FWIW it's a defense in depth measure[1] [...] [1] Consider an attacker with some ability to fabricate convincing signatures. Define convincing signatures. If anyone were to be able to create convincing gpg singatures of Apache releases, then this... Downloading the signature from a trusted server means that such an attacker would need to replace an existing signature on secure hardware without detection. is moot anyway, the lesser problem to be concerned about. And this btw. is not any different than to download the torrent via https. So it is not a matter of infrastructure, but a matter of policy. Where's the URL for this policy? I didn't mean to imply there was a set-in-stone policy already. What I meant was that it is up to the project to decide whether torrents are used or not, that the technical implementation of using torrents is so simple that apache infrastructure is not needed at all. You want torrents, you got torrents. You don't want them, you just don't use them. (Of course I don't know whether Apache as a whole has a written policy or guidelines wrt. using torrents, but I don't think there is one) ciao Christian
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
On Sun, Oct 23, 2011 at 9:08 PM, Marcus (OOo) marcus.m...@wtnet.de wrote: @List: Has anybody an idea about where to host this service? It doesn't need to be necessarily inside the ASF. The Pirate Bay? Lend them some legitimacy? :) More seriously, on the risk of being off-topic, has torrenting been considered for downloads? Don
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
On 25 Oct 2011, at 00:23, Donald Whytock wrote: On Sun, Oct 23, 2011 at 9:08 PM, Marcus (OOo) marcus.m...@wtnet.de wrote: @List: Has anybody an idea about where to host this service? It doesn't need to be necessarily inside the ASF. The Pirate Bay? Lend them some legitimacy? :) More seriously, on the risk of being off-topic, has torrenting been considered for downloads? It's been a feature of OOo for years: http://distribution.openoffice.org/p2p/ and is now also a feature at LibreOffice: http://www.libreoffice.org/download So I think that's a big yes and the question is how we keep it going... S.
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Am 10/25/2011 12:23 AM, schrieb Donald Whytock: On Sun, Oct 23, 2011 at 9:08 PM, Marcus (OOo)marcus.m...@wtnet.de wrote: @List: Has anybody an idea about where to host this service? It doesn't need to be necessarily inside the ASF. The Pirate Bay? Lend them some legitimacy? :) More seriously, on the risk of being off-topic, has torrenting been considered for downloads? Yes, torrents are already used. However, I would like to have something that we have under our own controal when we initializue a master server to distribute our release files. Marcus
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
On Mon, Oct 24, 2011 at 6:34 PM, Simon Phipps si...@webmink.com wrote: On 25 Oct 2011, at 00:23, Donald Whytock wrote: On Sun, Oct 23, 2011 at 9:08 PM, Marcus (OOo) marcus.m...@wtnet.de wrote: @List: Has anybody an idea about where to host this service? It doesn't need to be necessarily inside the ASF. The Pirate Bay? Lend them some legitimacy? :) More seriously, on the risk of being off-topic, has torrenting been considered for downloads? It's been a feature of OOo for years: http://distribution.openoffice.org/p2p/ and is now also a feature at LibreOffice: http://www.libreoffice.org/download So I think that's a big yes and the question is how we keep it going... I'd actually meant (and probably should have said) whether it has been considered by ASF. Make every mirror a seed and all that. Or, like in this case, have people willing to volunteer bits of space on a short-term basis. I didn't see anything about P2P at http://apache.org/dyn/closer.cgi. Does anyone know if ASF has any capability for this?
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Donald Whytock wrote on Mon, Oct 24, 2011 at 18:53:05 -0400: I didn't see anything about P2P at http://apache.org/dyn/closer.cgi. Does anyone know if ASF has any capability for this? Not to my knowledge.
RE: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
I read somewhere, and I don't know where, that ASF did not want torrents to be used. I'm guessing that the issue is related to ensuring the integrity and authenticity of packaged releases. Of course, one cannot prevent folks from torrenting binaries, but it is not something ASF wanted to seed. I may have dreamed it or I am mixing this up with something else. Sorry for not looking around more closely, have a household IT emergency. - Dennis -Original Message- From: Daniel Shahaf [mailto:d...@daniel.shahaf.name] Sent: Monday, October 24, 2011 16:06 To: Donald Whytock Cc: ooo-dev@incubator.apache.org Subject: Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance Donald Whytock wrote on Mon, Oct 24, 2011 at 18:53:05 -0400: I didn't see anything about P2P at http://apache.org/dyn/closer.cgi. Does anyone know if ASF has any capability for this? Not to my knowledge.
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Hi Marcus, hi list, On Sat, Oct 22, 2011 at 01:15:34 +0200, Marcus (OOo) wrote: Hi Peter, it seems the end is near and the download.services.openoffice.org host on Oracle side with our Mirrorbrain instance will be shutdown in one week. Okay... Little question, wouldn't it make sense, at this point, to take the download.s.o.o VM and move it somewhere else? I don't have the disk space myself, but maybe somebody else has? Perhaps the ASF can provide the space? (Other than disk space, the requirements are pretty modest.) Then the entire download service could continue to run, with very low effort, and without the need to build everything from scratch for now. As we cannot buildup a solution that is running, tested and long-term-proven in a few days I would like to ask you if we can switch to your openoffice.mirrorbrain.org instance as long as we have no other solution at hand. In the past it was a very reliable host that we have used when there were outages (mostly unplanned as you know) and doesn't resulted in a significant higher load on the host. So, it would help us very much to have a stable download section until we have an own solution here at ASF. Yes, please use my host for now! Thanks in advance and have a nice, sunny weekend. Thanks a lot. Same to you all, Peter pgpAYZu5yVeZd.pgp Description: PGP signature
Re: Shutdown of the download.services.openoffice.org host and its Mirrorbrain instance
Am 10/23/2011 11:50 PM, schrieb Peter Pöml: On Sat, Oct 22, 2011 at 01:15:34 +0200, Marcus (OOo) wrote: Hi Peter, it seems the end is near and the download.services.openoffice.org host on Oracle side with our Mirrorbrain instance will be shutdown in one week. Okay... Little question, wouldn't it make sense, at this point, to take the download.s.o.o VM and move it somewhere else? I don't have the disk space myself, but maybe somebody else has? Perhaps the ASF can provide the space? (Other than disk space, the requirements are pretty modest.) Then the entire download service could continue to run, with very low effort, and without the need to build everything from scratch for now. The problem is that the ASF do not want to host and provide services of special software for single projects. I can understand this as even the ASF infra is a team of volunteers and their time is limited as it is for all others. Furthermore, I don't know details about the VM setup and where it's detailed located. Do you? However, I like your idea. It requires indeed only a bit diskspace and internet access. The maintainance could be done by us, the project members. If you would share a bit of your knowledge then I could takeover the admin role. ;-) @List: Has anybody an idea about where to host this service? It doesn't need to be necessarily inside the ASF. As we cannot buildup a solution that is running, tested and long-term-proven in a few days I would like to ask you if we can switch to your openoffice.mirrorbrain.org instance as long as we have no other solution at hand. In the past it was a very reliable host that we have used when there were outages (mostly unplanned as you know) and doesn't resulted in a significant higher load on the host. So, it would help us very much to have a stable download section until we have an own solution here at ASF. Yes, please use my host for now! Great, thanks a lot. Thanks in advance and have a nice, sunny weekend. Thanks a lot. Same to you all, Marcus
