Re: [OSM-talk] Offline Dump of the Wiki
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Thanks a lot. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) iEYEAREIAAYFAkuQ5JIACgkQalWTFLzqsCsHdQCgszezvwZTZKtgp31KIg2tes7n 2tQAoIwL9ykYTPgB3+/aAcjwKq+Y3Gu5 =vAcn -END PGP SIGNATURE- ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
OSM Wiki dump is now available. http://dump.wiki.openstreetmap.org/ -current is only current page revisions. Done daily -files is only currently in usage images/pdf etc. Done daily. -full is full page history. Done weekly. Missing! http://commons.wikimedia.org/ embedded images. / Grant On 16 November 2009 11:57, Alexander Menk wrote: > Hi! > > how can I get a offline version of the wiki? > > Alexander > > > ___ > talk mailing list > talk@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
Tom Hughes wrote: > On 17/11/09 07:30, Alexander Menk wrote: > >> in case I do a wget of the wiki (of course excluding history and stuff) >> --- I would provide it as a zip archive for download... > > Please do not do anything as daft as trying to mirror the wiki with > wget. The poor machine can barely cope with serving real users without > also having to handle that. > > Tom > so what shall I do? We have a really bad internet connection here (Ethiopia), so without a offline Wiki it's not easy to contribute to OSM. Alexander ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
Kate wrote: > On Tue, Nov 17, 2009 at 5:28 AM, Tom Hughes wrote: >> Well who did the board ask to undertake this task? They can agree all >> they like, but unless they ask somebody to do it nothing much is going >> to happen. >> >> Probably the best way to do it will be to use the mediawiki dump tools >> but Grant probably knows more about that than me. We'll need to check >> that they don't leak any sensitive information, but I assume they won't >> as wikipedia publish dumps made that way. The raw database definitely >> does have sensitive information so can't be made available as is. >> >> I can't for the life of me think what sort of research would require a >> copy of the wiki though. >> >> Tom > > In the maintenance folder of the mediawiki installation, there is a > dumpBackup.php script: > > php maintenance/dumpBackup.php --full >20091117-osmwiki.xml > > This produces an xml dump that excludes all the sensitive information > about user accounts, and also excludes logs (e.g. deletion actions). > This is what Wikimedia does to provide database dumps. > > The importDump.php script in the maintenance folder can be used to put > the content into another wiki. > > I don't think a mirror of the wiki is needed and wouldn't edit on a > copy of the wiki, but suppose a copy would be useful to read and have > if off-line. > But what about the images? Alex ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
Tom Hughes wrote: > I can't for the life of me think what sort of research would require > a copy of the wiki though. I believe the project in question is known as "Infinite Number Of Crowdsourced Monkeys". The project methodology can be summarised as follows: Sit 5,000 monkeys in front of typewriters. The typewriters are standard Olivetti issue, rescued from a landfill in the Alto Adige area, but have been customised with three extra buttons: "I approve of this proposal", "I disapprove of this proposal", "This proposal has not been formulated according to the rules". Assess how long it takes for the monkeys to do any of the following: a) break the typewriters due to repeatedly hammering the same page of text b) throw the typewriters at each other c) gang up on any monkey who is not using the three extra buttons in the prescribed fashion d) demand that the typewriter technology is improved, because clearly that's what's stopping any useful output from being produced The original assessment criteria also included e) produce any worthwhile content, and f) go outside and do something useful. e) has been removed after a pilot experiment resulted in identical meaning from a week's worth of monkeys and a 'control' comprising 5,000 pages of lorem ipsum. Similarly, f) was abandoned after no monkey even made a single step towards the door during the pilot week, all of them remaining transfixed on their typewriters. The door has now been firmly bolted from the outside to stop any of the monkeys escaping. cheers Richard -- View this message in context: http://old.nabble.com/Offline-Dump-of-the-Wiki-tp26370710p26388338.html Sent from the OpenStreetMap - General mailing list archive at Nabble.com. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
On Tue, Nov 17, 2009 at 5:28 AM, Tom Hughes wrote: > > Well who did the board ask to undertake this task? They can agree all > they like, but unless they ask somebody to do it nothing much is going > to happen. > > Probably the best way to do it will be to use the mediawiki dump tools > but Grant probably knows more about that than me. We'll need to check > that they don't leak any sensitive information, but I assume they won't > as wikipedia publish dumps made that way. The raw database definitely > does have sensitive information so can't be made available as is. > > I can't for the life of me think what sort of research would require a > copy of the wiki though. > > Tom In the maintenance folder of the mediawiki installation, there is a dumpBackup.php script: php maintenance/dumpBackup.php --full >20091117-osmwiki.xml This produces an xml dump that excludes all the sensitive information about user accounts, and also excludes logs (e.g. deletion actions). This is what Wikimedia does to provide database dumps. The importDump.php script in the maintenance folder can be used to put the content into another wiki. I don't think a mirror of the wiki is needed and wouldn't edit on a copy of the wiki, but suppose a copy would be useful to read and have if off-line. -Kate > > -- > Tom Hughes (t...@compton.nu) > http://www.compton.nu/ > > ___ > talk mailing list > talk@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
Tom Hughes wrote: > On 17/11/09 07:30, Alexander Menk wrote: > >> in case I do a wget of the wiki (of course excluding history and stuff) >> --- I would provide it as a zip archive for download... > > Please do not do anything as daft as trying to mirror the wiki with > wget. The poor machine can barely cope with serving real users without > also having to handle that. the offline copy will serve real users... The idea is, if there is one carefully done offline copy of the wiki, less people would have to use the main wiki. Alexander ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
On Tuesday 17 Nov 2009 4:09:09 pm Peter Körner wrote: > > someone researching social networking ? > > In the osm wiki? for sure? > research institutes rarely require a reason for wanting anything -- regards Kenneth Gonsalves Senior Project Officer NRC-FOSS http://nrcfosshelpline.in/web/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
Tom Hughes wrote: > On 17/11/09 10:23, Simone Cortesi wrote: >> On Tue, Nov 17, 2009 at 11:17, Stefan de Konink wrote: >> >>> Can we get a url to wget the nightly backup from this separate physical >>> location to not disturb the active wiki, but still be able to serve a >>> wiki that is a day old? >> Some months ago we were asked by a research institute if we could >> provide a full copy of the wiki. The board did agree on this being >> available to the research institute. But the dump has not been made >> available to them yet. > > Well who did the board ask to undertake this task? They can agree all > they like, but unless they ask somebody to do it nothing much is going > to happen. > > Probably the best way to do it will be to use the mediawiki dump tools > but Grant probably knows more about that than me. We'll need to check > that they don't leak any sensitive information, but I assume they won't > as wikipedia publish dumps made that way. The raw database definitely > does have sensitive information so can't be made available as is. > > I can't for the life of me think what sort of research would require a > copy of the wiki though. If a suitable backup is available, then we can provide mirror sites. The problem of cause is that editing needs to be restricted to the main site, but even that can be catered for with care -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
> someone researching social networking ? In the osm wiki? for sure? Peter ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
s. > > I can't for the life of me think what sort of research would require a > copy of the wiki though. > > Tom > someone researching social networking ? ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
On 17/11/09 10:23, Simone Cortesi wrote: > On Tue, Nov 17, 2009 at 11:17, Stefan de Konink wrote: > >> Can we get a url to wget the nightly backup from this separate physical >> location to not disturb the active wiki, but still be able to serve a >> wiki that is a day old? > > Some months ago we were asked by a research institute if we could > provide a full copy of the wiki. The board did agree on this being > available to the research institute. But the dump has not been made > available to them yet. Well who did the board ask to undertake this task? They can agree all they like, but unless they ask somebody to do it nothing much is going to happen. Probably the best way to do it will be to use the mediawiki dump tools but Grant probably knows more about that than me. We'll need to check that they don't leak any sensitive information, but I assume they won't as wikipedia publish dumps made that way. The raw database definitely does have sensitive information so can't be made available as is. I can't for the life of me think what sort of research would require a copy of the wiki though. Tom -- Tom Hughes (t...@compton.nu) http://www.compton.nu/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
On Tue, Nov 17, 2009 at 11:17, Stefan de Konink wrote: > On Tue, 17 Nov 2009, Tom Hughes wrote: > >> What does our backup strategy have to do with the question of whether or >> not it's a good idea to mirror the wiki with wget? >> >> For what it's worth, the wiki is backed up nightly to a separate >> physical location. > > Can we get a url to wget the nightly backup from this separate physical > location to not disturb the active wiki, but still be able to serve a > wiki that is a day old? Some months ago we were asked by a research institute if we could provide a full copy of the wiki. The board did agree on this being available to the research institute. But the dump has not been made available to them yet. -S ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
On Tue, 17 Nov 2009, Tom Hughes wrote: > What does our backup strategy have to do with the question of whether or > not it's a good idea to mirror the wiki with wget? > > For what it's worth, the wiki is backed up nightly to a separate > physical location. Can we get a url to wget the nightly backup from this separate physical location to not disturb the active wiki, but still be able to serve a wiki that is a day old? Stefan ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
On 17/11/09 09:27, Lester Caine wrote: > Tom Hughes wrote: > >> Please do not do anything as daft as trying to mirror the wiki with >> wget. The poor machine can barely cope with serving real users without >> also having to handle that. > > What database is the wiki running on? My own sites I simply have a backup of > that running and rsync the backup along with the images directory to the > backup > machine. Very little load on either machine What does our backup strategy have to do with the question of whether or not it's a good idea to mirror the wiki with wget? For what it's worth, the wiki is backed up nightly to a separate physical location. Tom -- Tom Hughes (t...@compton.nu) http://www.compton.nu/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
Tom Hughes wrote: > On 17/11/09 07:30, Alexander Menk wrote: > >> in case I do a wget of the wiki (of course excluding history and stuff) >> --- I would provide it as a zip archive for download... > > Please do not do anything as daft as trying to mirror the wiki with > wget. The poor machine can barely cope with serving real users without > also having to handle that. What database is the wiki running on? My own sites I simply have a backup of that running and rsync the backup along with the images directory to the backup machine. Very little load on either machine -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
On 17/11/09 07:30, Alexander Menk wrote: > in case I do a wget of the wiki (of course excluding history and stuff) > --- I would provide it as a zip archive for download... Please do not do anything as daft as trying to mirror the wiki with wget. The poor machine can barely cope with serving real users without also having to handle that. Tom -- Tom Hughes (t...@compton.nu) http://www.compton.nu/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
Hi! in case I do a wget of the wiki (of course excluding history and stuff) --- I would provide it as a zip archive for download... Alex Rob wrote: > even better, how can we mirror the wiki and have lightning fast local mirrors > ;) > > 2009/11/16 Alexander Menk > : >> Hi! >> >> how can I get a offline version of the wiki? >> >> Alexander >> >> ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Offline Dump of the Wiki
even better, how can we mirror the wiki and have lightning fast local mirrors ;) 2009/11/16 Alexander Menk : > Hi! > > how can I get a offline version of the wiki? > > Alexander > > > ___ > talk mailing list > talk@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk > > ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Offline Dump of the Wiki
Hi! how can I get a offline version of the wiki? Alexander ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
