Re: [Foundation-l] It Is not Us
Hoi, I have read the replies that are against social networking functionality. In my opinion you are all missing the point. Our projects are crowd sourced projects and we do not support collaboration, we do not support special projects. We need to. Social networking in our context will not be a Facebook, a Twitter or an IRC. It will have the parts that we need and it will support our activities. Thanks, GerardM http://ultimategerardm.blogspot.com/2011/06/in-defence-of-social-networks.html On 27 June 2011 18:24, Nathan nawr...@gmail.com wrote: On Mon, Jun 27, 2011 at 11:43 AM, Gerard Meijssen gerard.meijs...@gmail.com wrote: Hoi. Wikipedia should be more like a social network. It provides us with the opportunity to reach out to people when we want to crowd source some activity. We have a problem in retaining people particular newbies. When we show a social side to our work on open content (not only encyclopaedic content) we stand a better chance we are likely to do better. Thanks, GerardM That's an interesting theory. Wikipedia is sort of the epitome of a social enterprise, and all of the good and the bad in the project can be traced to its social nature. Trying to make it more like a social network can only be interpreted as making it more like some other social network, perhaps by integrating purely social mechanisms a la Facebook. Of course, that could either help or hinder, with no way to know for sure in advance; perhaps encouraging more social interaction would exacerbate and personalize the disputes and conflicts that drive people away. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] [Wiki-research-l] Wikipedia dumps downloader
Can you share your script with us? 2011/6/27 Platonides platoni...@gmail.com emijrp wrote: Hi SJ; You know that that is an old item in our TODO list ; ) I heard that Platonides developed a script for that task long time ago. Platonides, are you there? Regards, emijrp Yes, I am. :) ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] [Xmldatadumps-l] Wikipedia dumps downloader
On 06/27/2011 01:07 PM, emijrp wrote: Yes, a distributed project would be probably the best solution, but it is not easy to develop, unless you use a library like bittorrent, or similar and you have many peers. Althought most of the people don't seed the files long time, so sometimes is better to depend on a few committed persons than a big but ephemeral crowd. If you create a way to sync images partially (let's say, the base letters inside of the directory structure), a couple of dozens of Wikimedians could create a couple of dozens of backups. I could host on my personal computer ~500GB, which approximates one letter. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
[Foundation-l] Invitation to connect on LinkedIn
LinkedIn Mohamed Ibrahim requested to add you as a connection on LinkedIn: -- Srikant, I'd like to add you to my professional network on LinkedIn. - Mohamed Accept invitation from Mohamed Ibrahim http://www.linkedin.com/e/-t9tct-gpglc3s9-54/kEGYngBeZsro1F93qKeRGrJetgIrKFfJd2o_ToVctvnatuZm/blk/I54599208_75/pmpxnSRJrSdvj4R5fnhv9ClRsDgZp6lQs6lzoQ5AomZIpn8_djtve30OejARd3l9bP0Td6F5r5dEbPgMdzAPej4ScPcLrCBxbOYWrSlI/EML_comm_afe/ View invitation from Mohamed Ibrahim http://www.linkedin.com/e/-t9tct-gpglc3s9-54/kEGYngBeZsro1F93qKeRGrJetgIrKFfJd2o_ToVctvnatuZm/blk/I54599208_75/0RdRYUc38VejkQdkALqnpPbOYWrSlI/svi/ -- DID YOU KNOW you can showcase your professional knowledge on LinkedIn to receive job/consulting offers and enhance your professional reputation? Posting replies to questions on LinkedIn Answers puts you in front of the world's professional community. http://www.linkedin.com/e/-t9tct-gpglc3s9-54/abq/inv-24/ -- (c) 2011, LinkedIn Corporation ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Invitation to connect on LinkedIn
#mega #Fail dunno why the list is saved as a contact with that name Can we add something like Invitation to connect on LinkedIn in the filter so this kind of e-mail will be automatically discarded? Regards, -- Tanvir Rahman [[User:Wikitanvir]] On Wikimedia Projects ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Invitation to connect on LinkedIn
basically, how can someone register on linkedIn with foundation-l address? it was a linked account that caused this problem On 28 June 2011 11:42, Tanvir Rahman wikitan...@gmail.com wrote: #mega #Fail dunno why the list is saved as a contact with that name Can we add something like Invitation to connect on LinkedIn in the filter so this kind of e-mail will be automatically discarded? Regards, -- Tanvir Rahman [[User:Wikitanvir]] On Wikimedia Projects ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l -- Mohamed Ibrahim Architect Cell : +966 54 4680745 E-mail: arch.m.ibra...@gmail.com http://eg.linkedin.com/in/mohamedibrahims ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Invitation to connect on LinkedIn
Perhaps one of the list admins could at least build in a filter for the linked in emails :) Possibly file a request with LinkedIn for change of the account details too? Best, Lodewijk 2011/6/28 Mohamed Ibrahim mido.archit...@gmail.com basically, how can someone register on linkedIn with foundation-l address? it was a linked account that caused this problem On 28 June 2011 11:42, Tanvir Rahman wikitan...@gmail.com wrote: #mega #Fail dunno why the list is saved as a contact with that name Can we add something like Invitation to connect on LinkedIn in the filter so this kind of e-mail will be automatically discarded? Regards, -- Tanvir Rahman [[User:Wikitanvir]] On Wikimedia Projects ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l -- Mohamed Ibrahim Architect Cell : +966 54 4680745 E-mail: arch.m.ibra...@gmail.com http://eg.linkedin.com/in/mohamedibrahims ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] It Is not Us
Hoi, I have read the replies that are against social networking functionality. In my opinion you are all missing the point. Our projects are crowd sourced projects and we do not support collaboration, we do not support special projects. We need to. Social networking in our context will not be a Facebook, a Twitter or an IRC. It will have the parts that we need and it will support our activities. Thanks, GerardM I always go back to the userbox controversy when I think about this. What would we look like if we had not only embraced userboxes but created a complex system of user categories based on them? Fred ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] It Is not Us
On 28 June 2011 08:35, Gerard Meijssen gerard.meijs...@gmail.com wrote: Hoi, I have read the replies that are against social networking functionality. In my opinion you are all missing the point. Our projects are crowd sourced projects and we do not support collaboration, we do not support special projects. We need to. Yeah! Special projects with a narrower focus would be great, how about giving them a catchy name like WikiProjects. Maybe we could give every article a talk page for users to collaborate on. Heck, let's go mad and give users their own talk pages too! Now if only there was some protocol for real time chats we could use... Social networking in our context will not be a Facebook, a Twitter or an IRC. It will have the parts that we need and it will support our activities. Thanks, I'm all for improving the interface around these things, but exactly what functionality are you asking for that we don't already have? Pete / the wub On 27 June 2011 18:24, Nathan nawr...@gmail.com wrote: On Mon, Jun 27, 2011 at 11:43 AM, Gerard Meijssen gerard.meijs...@gmail.com wrote: Hoi. Wikipedia should be more like a social network. It provides us with the opportunity to reach out to people when we want to crowd source some activity. We have a problem in retaining people particular newbies. When we show a social side to our work on open content (not only encyclopaedic content) we stand a better chance we are likely to do better. Thanks, GerardM That's an interesting theory. Wikipedia is sort of the epitome of a social enterprise, and all of the good and the bad in the project can be traced to its social nature. Trying to make it more like a social network can only be interpreted as making it more like some other social network, perhaps by integrating purely social mechanisms a la Facebook. Of course, that could either help or hinder, with no way to know for sure in advance; perhaps encouraging more social interaction would exacerbate and personalize the disputes and conflicts that drive people away. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] [Xmldatadumps-l] Wikipedia dumps downloader
On Tue, Jun 28, 2011 at 10:12, Milos Rancic mill...@gmail.com wrote: On 06/27/2011 01:07 PM, emijrp wrote: If you create a way to sync images partially (let's say, the base letters inside of the directory structure), a couple of dozens of Wikimedians could create a couple of dozens of backups. I could host on my personal computer ~500GB, which approximates one letter. Some chapters have servers with storage space, too. g ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Invitation to connect on LinkedIn
I have added in a filter which should stop these from going to the list. User:AlexandrDmitri on all Wikimedia Projects 2011/6/28 Lodewijk lodew...@effeietsanders.org Perhaps one of the list admins could at least build in a filter for the linked in emails :) Possibly file a request with LinkedIn for change of the account details too? Best, Lodewijk 2011/6/28 Mohamed Ibrahim mido.archit...@gmail.com basically, how can someone register on linkedIn with foundation-l address? it was a linked account that caused this problem On 28 June 2011 11:42, Tanvir Rahman wikitan...@gmail.com wrote: #mega #Fail dunno why the list is saved as a contact with that name Can we add something like Invitation to connect on LinkedIn in the filter so this kind of e-mail will be automatically discarded? Regards, -- Tanvir Rahman [[User:Wikitanvir]] On Wikimedia Projects ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l -- Mohamed Ibrahim Architect Cell : +966 54 4680745 E-mail: arch.m.ibra...@gmail.com http://eg.linkedin.com/in/mohamedibrahims ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Invitation to connect on LinkedIn
You don't have to have an account on LinkedIn to receive these emails. We were getting them sent to OTRS email addresses all the time before I asked LinkedIn to block emails to the addresses. Had to go through customer service because they don't provide an opt-out link in the emails they send. We still get some to a few addresses I asked to opt out of. I thought I also asked the list to be opted out as well, but potentially they received a bounce email back and so disregarded the ticket. The main concern is people need to stop uploading their contacts lists and then telling LinkedIn to invite *everyone* in it. Take the time to select actual individuals rather than spamming everyone. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
[Foundation-l] BDS (hopefully) FUD?
At least 1 BDS Activist has been running around on twitter telling people not to go to wikimania. http://twitter.com/_C Our article on BDS: http://en.wikipedia.org/wiki/Boycott,_Divestment_and_Sanctions BDS view and action on wikimania: http://www.alternativenews.org/english/index.php/topics/economy-of-the-occupation/3439-bds-action-tell-wikimedia-not-to-hold-2011-annual-conference-in-israel- In particular, they state: On the information page for the upcoming conference, the organizers also say, ???Haifa is also easily accessible to the Palestinian community, which is often left out of conferences such as Wikimania due to special difficulties. This community has a high rate of Internet usage but little first-hand acquaintance with Wikipedia and other Wikimedia projects.??? The statement, however, is not true. Participation for individuals from Gaza and the West Bank is impossible, unless they receive special permits from the Israeli military to enter Israel in order to attend. To be able to actively counter this claim, can we confirm that all palestinian wikimedians who want to come are indeed coming? sincerely, Kim Bruning -- [Non-pgp mail clients may show pgp-signature as attachment] gpg (www.gnupg.org) Fingerprint for key FEF9DD72 5ED6 E215 73EE AD84 E03A 01C5 94AC 7B0E FEF9 DD72 ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] [Wiki-research-l] Wikipedia dumps downloader
Hi; @Derrick: I don't trust Amazon. Really, I don't trust Wikimedia Foundation either. They can't and/or they don't want to provide image dumps (what is worst?). Community donates images to Commons, community donates money every year, and now community needs to develop a software to extract all the images and packed them, and of course, host them in a permanent way. Crazy, right? @Milos: Instead of spliting image dump using the first letter of filenames, I thought about spliting using the upload date (-MM-DD). So, first chunks (2005-01-01) will be tiny, and recent ones of several GB (a single day). Regards, emijrp 2011/6/28 Derrick Coetzee dcoet...@eecs.berkeley.edu As a Commons admin I've thought a lot about the problem of distributing Commons dumps. As for distribution, I believe BitTorrent is absolutely the way to go, but the Torrent will require a small network of dedicated permaseeds (servers that seed indefinitely). These can easily be set up at low cost on Amazon EC2 small instances - the disk storage for the archives is free, since small instances include a large (~120 GB) ephemeral storage volume at no additional cost, and the cost of bandwidth can be controlled by configuring the BitTorrent client with either a bandwidth throttle or a transfer cap (or both). In fact, I think all Wikimedia dumps should be available through such a distribution solution, just as all Linux installation media are today. Additionally, it will be necessary to construct (and maintain) useful subsets of Commons media, such as all media used on the English Wikipedia, or thumbnails of all images on Wikimedia Commons, of particular interest to certain content reusers, since the full set is far too large to be of interest to most reusers. It's on this latter point that I want your feedback: what useful subsets of Wikimedia Commons does the research community want? Thanks for your feedback. --=20 Derrick Coetzee User:Dcoetzee, English Wikipedia and Wikimedia Commons administrator http://www.eecs.berkeley.edu/~dcoetzee/ ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] [Wiki-research-l] Wikipedia dumps downloader
On 06/28/2011 07:21 PM, emijrp wrote: @Milos: Instead of spliting image dump using the first letter of filenames, I thought about spliting using the upload date (-MM-DD). So, first chunks (2005-01-01) will be tiny, and recent ones of several GB (a single day). That would be better, indeed! And you could create a wiki page where people like myself would coordinate backup: we should cover the backup once, then we could create more. For example, I can cover every Nth day in month. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] BDS (hopefully) FUD?
When you use Wikimania and Isreal or Wiki(M/P)edia with the word Isreal there will be 3 bots telling you not to go :-( 2011/6/28, Kim Bruning k...@bruning.xs4all.nl: At least 1 BDS Activist has been running around on twitter telling people not to go to wikimania. http://twitter.com/_C Our article on BDS: http://en.wikipedia.org/wiki/Boycott,_Divestment_and_Sanctions BDS view and action on wikimania: http://www.alternativenews.org/english/index.php/topics/economy-of-the-occupation/3439-bds-action-tell-wikimedia-not-to-hold-2011-annual-conference-in-israel- In particular, they state: On the information page for the upcoming conference, the organizers also say, “Haifa is also easily accessible to the Palestinian community, which is often left out of conferences such as Wikimania due to special difficulties. This community has a high rate of Internet usage but little first-hand acquaintance with Wikipedia and other Wikimedia projects.” The statement, however, is not true. Participation for individuals from Gaza and the West Bank is impossible, unless they receive special permits from the Israeli military to enter Israel in order to attend. To be able to actively counter this claim, can we confirm that all palestinian wikimedians who want to come are indeed coming? sincerely, Kim Bruning -- [Non-pgp mail clients may show pgp-signature as attachment] gpg (www.gnupg.org) Fingerprint for key FEF9DD72 5ED6 E215 73EE AD84 E03A 01C5 94AC 7B0E FEF9 DD72 -- Verzonden vanaf mijn mobiele apparaat Kind regards, Huib Laurens WickedWay.nl Webhosting the wicked way. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] BDS (hopefully) FUD?
On Tue, Jun 28, 2011 at 06:13:54PM +0200, Kim Bruning wrote: To be able to actively counter this claim, can we confirm that all palestinian wikimedians who want to come are indeed coming? I've since done a little more research. I'm moving this question to wikimania-l, in a better focussed and restated version. sincerely, Kim Bruning -- [Non-pgp mail clients may show pgp-signature as attachment] gpg (www.gnupg.org) Fingerprint for key FEF9DD72 5ED6 E215 73EE AD84 E03A 01C5 94AC 7B0E FEF9 DD72 ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] BDS (hopefully) FUD?
On Tue, Jun 28, 2011 at 08:06:43PM +0200, Huib Laurens wrote: When you use Wikimania and Isreal or Wiki(M/P)edia with the word Isreal there will be 3 bots telling you not to go :-( Which bots are they? can you link to their twitter pages? sincerely, Kim Bruning -- [Non-pgp mail clients may show pgp-signature as attachment] gpg (www.gnupg.org) Fingerprint for key FEF9DD72 5ED6 E215 73EE AD84 E03A 01C5 94AC 7B0E FEF9 DD72 ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] BDS (hopefully) FUD?
Talking about that: Wikimania has a official hasgtag? _ *Béria Lima* http://wikimedia.pt/(351) 925 171 484 *Imagine um mundo onde é dada a qualquer pessoa a possibilidade de ter livre acesso ao somatório de todo o conhecimento humano. É isso o que estamos a fazer http://wikimediafoundation.org/wiki/Nossos_projetos.* 2011/6/28 Kim Bruning k...@bruning.xs4all.nl On Tue, Jun 28, 2011 at 06:13:54PM +0200, Kim Bruning wrote: To be able to actively counter this claim, can we confirm that all palestinian wikimedians who want to come are indeed coming? I've since done a little more research. I'm moving this question to wikimania-l, in a better focussed and restated version. sincerely, Kim Bruning -- [Non-pgp mail clients may show pgp-signature as attachment] gpg (www.gnupg.org) Fingerprint for key FEF9DD72 5ED6 E215 73EE AD84 E03A 01C5 94AC 7B0E FEF9 DD72 ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] [Wiki-research-l] Wikipedia dumps downloader
emijrp wrote: Hi; @Derrick: I don't trust Amazon. I disagree. Note that we only need them to keep a redundant copy of a file. If they tried to tamper the file we could detect it with the hashes (which should be properly secured, that's no problem). I'd like having the hashes for the xml dumps content instead of the compressed one, though, so it could be easily stored with better compression without weakening the integrity check. Really, I don't trust Wikimedia Foundation either. They can't and/or they don't want to provide image dumps (what is worst?). Wikimedia Foundation has provided image dumps several times in the past, and also rsync3 access to some individuals so that they could clone it. It's like the enwiki history dump. An image dump is complex, and even less useful. Community donates images to Commons, community donates money every year, and now community needs to develop a software to extract all the images and packed them, There's no *need* for that. In fact, such script would be trivial from the toolserver. and of course, host them in a permanent way. Crazy, right? WMF also tries hard to not lose images. We want to provide some redundance on our own. That's perfectly fine, but it's not a requirement. Consider that WMF could be automatically deleting page history older than a month, or images not used on any article. *That* would be a real problem. @Milos: Instead of spliting image dump using the first letter of filenames, I thought about spliting using the upload date (-MM-DD). So, first chunks (2005-01-01) will be tiny, and recent ones of several GB (a single day). Regards, emijrp I like that idea since it means the dumps are static. They could be placed in tape inside a safe and not needed to be taken out unless data loss arises. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] [Wiki-research-l] Wikipedia dumps downloader
2011/6/28 Platonides platoni...@gmail.com emijrp wrote: Hi; @Derrick: I don't trust Amazon. I disagree. Note that we only need them to keep a redundant copy of a file. If they tried to tamper the file we could detect it with the hashes (which should be properly secured, that's no problem). I didn't mean security problems. I meant just deleted files by weird terms of service. Commons hosts a lot of images which can be problematic, like nudes or copyrighted materials in some jurisdictions. They can deleted what they want and close every account they want, and we will lost the backups. Period. And we don't only need to keep a copy of every file. We need several copies everywhere, not only in the Amazon coolcloud. I'd like having the hashes for the xml dumps content instead of the compressed one, though, so it could be easily stored with better compression without weakening the integrity check. Really, I don't trust Wikimedia Foundation either. They can't and/or they don't want to provide image dumps (what is worst?). Wikimedia Foundation has provided image dumps several times in the past, and also rsync3 access to some individuals so that they could clone it. Ah, OK, that is enough (?). Then, you are OK with old-and-broken XML dumps, because people can slurp all the pages using an API scrapper. It's like the enwiki history dump. An image dump is complex, and even less useful. It is not complex, just resources consuming. If they need to buy another 10 TB of space and more CPU, they can. $16M were donated last year. They just need to put resources in relevant stuff. WMF always says we host the 5th website in the world, I say that they need to act like that. Less useful? I hope they don't need such a useless dump for recovering images, just like happened in the past. Community donates images to Commons, community donates money every year, and now community needs to develop a software to extract all the images and packed them, There's no *need* for that. In fact, such script would be trivial from the toolserver. Ah, OK, only people with toolserver account may have access to an image dump. And you say it is trivial from Toolserver and very complex from Wikimedia main servers. and of course, host them in a permanent way. Crazy, right? WMF also tries hard to not lose images. I hope that, but we remember a case of lost images. We want to provide some redundance on our own. That's perfectly fine, but it's not a requirement. That _is_ a requirement. We can't trust Wikimedia Foundation. They lost images. They have problems to generate English Wikipedia dumps and image dumps. They had a hardware failure some months ago in the RAID which hosts the XML dumps, and they didn't offer those dumps during months, while trying to fix the crash. Consider that WMF could be automatically deleting page history older than a month, or images not used on any article. *That* would be a real problem. You just don't understand how dangerous is the current status (and it was worst in the past). @Milos: Instead of spliting image dump using the first letter of filenames, I thought about spliting using the upload date (-MM-DD). So, first chunks (2005-01-01) will be tiny, and recent ones of several GB (a single day). Regards, emijrp I like that idea since it means the dumps are static. They could be placed in tape inside a safe and not needed to be taken out unless data loss arises. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] [Wiki-research-l] Wikipedia dumps downloader
emijrp wrote: I didn't mean security problems. I meant just deleted files by weird terms of service. Commons hosts a lot of images which can be problematic, like nudes or copyrighted materials in some jurisdictions. They can deleted what they want and close every account they want, and we will lost the backups. Period. Good point. And we don't only need to keep a copy of every file. We need several copies everywhere, not only in the Amazon coolcloud. Sure. Relying *just* on Amazon would be very bad. Wikimedia Foundation has provided image dumps several times in the past, and also rsync3 access to some individuals so that they could clone it. Ah, OK, that is enough (?). Then, you are OK with old-and-broken XML dumps, because people can slurp all the pages using an API scrapper. If all people that wants it can get it, then it's enough. Not so much in a timely manner, though, but that could be fixed. I'm quite confident that if rediris rang me tomorrow offering 20Tb for hosting commosns image dumps, that could be managed without too much problems. It's like the enwiki history dump. An image dump is complex, and even less useful. It is not complex, just resources consuming. If they need to buy another 10 TB of space and more CPU, they can. $16M were donated last year. They just need to put resources in relevant stuff. WMF always says we host the 5th website in the world, I say that they need to act like that. Less useful? I hope they don't need such a useless dump for recovering images, just like happened in the past. Yes, that seems sensible. You just need to convince them :) But note that they are already making another datacenter and developing a system with which they would keep a copy of every upload on both of them. They are not so mean. Community donates images to Commons, community donates money every year, and now community needs to develop a software to extract all the images and packed them, There's no *need* for that. In fact, such script would be trivial from the toolserver. Ah, OK, only people with toolserver account may have access to an image dump. And you say it is trivial from Toolserver and very complex from Wikimedia main servers. Come on. Making a script to dowload all images is trivial from the toolserver. It's just not so easy using the api. The complexity is for making a dump that *anyone* can download. And it's just resources, not technical. and of course, host them in a permanent way. Crazy, right? WMF also tries hard to not lose images. I hope that, but we remember a case of lost images. Yes. That's a reason for making copies, and I support that. But there's a difference between failures happen and WMF is not trying to keep copies. We want to provide some redundance on our own. That's perfectly fine, but it's not a requirement. That _is_ a requirement. We can't trust Wikimedia Foundation. They lost images. They have problems to generate English Wikipedia dumps and image dumps. They had a hardware failure some months ago in the RAID which hosts the XML dumps, and they didn't offer those dumps during months, while trying to fix the crash. You just don't understand how dangerous is the current status (and it was worst in the past). The big problem is its huge size. If it was 2MB everyone and his grandmother would keep a copy. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
[Foundation-l] IRC Bug Triage now
We'll be holding a public IRC bug triage in about 2-3 minutes on #wikimedia-dev for all who are interested. We will be using http://etherpad.wikimedia.org/BugTriage-2011-06 to keep notes as well. See you there! Mark. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
