Re: Wikipedia on Python
On Tue, 16 Oct 2018 at 20:11, Chris Angelico wrote: > > On Wed, Oct 17, 2018 at 5:05 AM Spencer Graves > wrote: > >Beyond that, I'd like to encourage people on this list to review > > the Wikipedia article on "Python (programming language)",[1] especially > > the claim that "a package is a Python module with an __path__ > > attribute", which I added on 2018-09-24 to help me understand the > > distinction. > > > > You're welcome to put whatever you like into Wikipedia, but > personally, I don't think that particular piece of terminology is all > that helpful to the typical reader. Technical distinctions aren't > important to someone who's trying to find out what Python's all about, > or why s/he should learn the language. Seconded. This is not useful at all on Wikipedia. I took the liberty to remove this paragraph, because I don’t think anyone would find it useful; in fact, it would only confuse people. Here’s a diff for anyone interested in the original content: https://en.wikipedia.org/w/index.php?title=Python_(programming_language)=prev=861064627 -- Chris Warrick <https://chriswarrick.com/> PGP: 5EAAEA16 -- https://mail.python.org/mailman/listinfo/python-list
RE: Wikipedia on Python
Not really having read the whole story here, just wanna say that Wikipedia articles already suffer from a tug-of-war between professionals and students. Don’t worsen the problem. Include lay explanation and go into the technical details in a natural progression. Don’t force the reader to make large perspective jumps or conceptual jumps or require them to approach the article from the perspective of the designer from one sentence to the next. Pick your audience first, then write. Wikipedia requires a bridge between student and professional. Write it like it’s a synopsis that goes into detail, not like a reference manual for the already-initiated reader. If you like to write like it’s a reference, then there are list metapages on Wikipedia that tend to be agreeable to this. peace Sent from Mail for Windows 10 From: Spencer Graves Sent: Tuesday, October 16, 2018 1:06 PM To: Léo El Amri; python-list@python.org Subject: Wikipedia on Python Thanks to Léo El Amri and Thomas Jollans for their quick and helpful replies to my question about "Package creation documentation". Beyond that, I'd like to encourage people on this list to review the Wikipedia article on "Python (programming language)",[1] especially the claim that "a package is a Python module with an __path__ attribute", which I added on 2018-09-24 to help me understand the distinction. That Wikipedia article has averaged over 6,000 views per day over the past 3 years. Therefore, any improvements will benefit lots of people. If you have suggestions for how the article might be improved, you can post them to the "Talk" page associated with that article or send them to me. If you are "autoconfirmed" with the Wikimedia system, you can make the changes yourself. Thanks, Spencer Graves [1] https://en.wikipedia.org/wiki/Python_(programming_language) On 2018-10-16 11:14, Léo El Amri wrote: > Hello Spencer, > > On 16/10/2018 17:15, Spencer Graves wrote: >>Where can I find a reasonable tutorial on how to create a Python >> package? > IMO, the best documentation about this is the tutorial: > https://docs.python.org/3/tutorial/modules.html#packages > >>According to the Python 3 Glossary, "a package is a Python module >> with an __path__ attribute."[1] > What you are looking at are the technical details of what a package is. > Incidentally, if you follow the tutorial, everything will get in-place. > >>I found "packaging.python.org", which recommends "Packaging Python >> Projects"[2] and "An Overview of Packaging for Python".[3] > packaging.python.org is centered on "How to install and distribute > Python packages (Or modules)" > > - Léo > -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Wikipedia on Python
On Wed, Oct 17, 2018 at 5:05 AM Spencer Graves wrote: >Beyond that, I'd like to encourage people on this list to review > the Wikipedia article on "Python (programming language)",[1] especially > the claim that "a package is a Python module with an __path__ > attribute", which I added on 2018-09-24 to help me understand the > distinction. > You're welcome to put whatever you like into Wikipedia, but personally, I don't think that particular piece of terminology is all that helpful to the typical reader. Technical distinctions aren't important to someone who's trying to find out what Python's all about, or why s/he should learn the language. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Wikipedia on Python
Thanks to Léo El Amri and Thomas Jollans for their quick and helpful replies to my question about "Package creation documentation". Beyond that, I'd like to encourage people on this list to review the Wikipedia article on "Python (programming language)",[1] especially the claim that "a package is a Python module with an __path__ attribute", which I added on 2018-09-24 to help me understand the distinction. That Wikipedia article has averaged over 6,000 views per day over the past 3 years. Therefore, any improvements will benefit lots of people. If you have suggestions for how the article might be improved, you can post them to the "Talk" page associated with that article or send them to me. If you are "autoconfirmed" with the Wikimedia system, you can make the changes yourself. Thanks, Spencer Graves [1] https://en.wikipedia.org/wiki/Python_(programming_language) On 2018-10-16 11:14, Léo El Amri wrote: Hello Spencer, On 16/10/2018 17:15, Spencer Graves wrote: Where can I find a reasonable tutorial on how to create a Python package? IMO, the best documentation about this is the tutorial: https://docs.python.org/3/tutorial/modules.html#packages According to the Python 3 Glossary, "a package is a Python module with an __path__ attribute."[1] What you are looking at are the technical details of what a package is. Incidentally, if you follow the tutorial, everything will get in-place. I found "packaging.python.org", which recommends "Packaging Python Projects"[2] and "An Overview of Packaging for Python".[3] packaging.python.org is centered on "How to install and distribute Python packages (Or modules)" - Léo -- https://mail.python.org/mailman/listinfo/python-list
Re: Scraping Wikipedia with Python
On Tue, Aug 11, 2009 at 8:53 PM, David C Ullrichdullr...@sprynet.com wrote: Try reading a little there! Starting there I went to http://en.wikipedia.org/wiki/Wikipedia:Creating_a_bot where I found a section on existing bots, comments on how the scraping is not what you want, and even a Python section with a link to something labelled PyWikipediaBot... Some information on using the PyWikipediaBot for scraping from someone who used to program on the bot (and occasionally still does): To make the framework work, you need to add a file user-config.py with the following contents: family = 'wikipedia' mylang = 'en' If you want to use the bot to also edit pages on wikipedia, you will have to add: usernames['wikipedia']['en'] = the username of your bot If you work on another language of course you use that language's abbreviation instead of en. The heart of the framework is the file wikipedia.py, you need to import that one. It contains two important classes: Page and Site, which represent a wikipedia page and the site as a whole, respectively. It is best to put your code in a try like this: try: mysite = wikipedia.getSite() your code here finally: wikipedia.stopme() The stopme() functionality has to do with the bot's behaviour to avoid over-feeding the server with requests. It has a certain time (default is 10 seconds) between two requests, but if you have several bots running, it will lengthen this time. stopme() tells that the bot is not running any more, so other runs are not delayed by it. wikipedia.getSite() gets the site object for your default site (if the settings above are chosen it is the English language Wikipedia). Still with me? Good, because now we get into the real programming. The Page class has as its __init__: def __init__(self, site, title, insite=None, defaultNamespace=0): site is here the wiki on which the page exists (usually this will be mysite, which is why I defined it above), title the title of the page. The optional parameters are for special usage. The Page class has a number of methods, which you can find in the file, but some of the most important are: page.title() - the title of the page page.site() - the wiki the page is on page.get() - the (wiki) text of the page page.put(text) - saves the page with 'text' as its new content. An important optional parameter is 'comment', which specifies the summary that is given with the change page.exists() - a boolean, true if the page exists, false otherwise page.linkedPages() - a list of Page objects, being the pages the page links to However, instead of page.get() it is advisable to use: wikipedia.getall(site,pages) with 'site' being a Site object (e.g. mysite) and pages a list (or more generally, iterable) of Page objects. It will get all pages in the list using a single call to the wiki, thus speeding up your bot and at the same time reducing its load on the wiki. Once a page has been loaded (either through get or through getall), subsequent calls to page.get() will not reload it. Thus, the normal way of working is to create a list of pages one is interested in, use getall (in groups of 60 or so) to load them, then use get to work with them. Another useful file in the framework is pagegenerators. It provides a number of generators that yield Page objects. Some interesting ones (check the code for the exact parameters): AllpagesPageGenerator: generates all pages of the wiki, alphabetically from a specified begin ReferringPageGenerator: all pages linking to a given page CategorizedPageGenerator: all pages in a given directory LinkedPageGenerator: all pages linked to from a given page Other generators are used by 'wrapping them around' a given generator. The most important of these is the PreloadingGenerator, which ensures that the page are preloaded (using wikipedia.getall) in groups. A simple way to use the bot framework to scrape all pages of the English Wikipedia (warning: This takes a few days!) would be: import wikipedia import pagegenerators basicgen = pagegenerators.AllpagesPageGenerator(includeredirects = False) generator = pagegenerators.PreloadingGenerator(basicgen, 200) for page in generator: title = page.title() text = page.get() do whatever you want with title and text -- André Engels, andreeng...@gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Scraping Wikipedia with Python
* Dotan Cohen (Tue, 11 Aug 2009 21:29:40 +0300) Wikipedia has an API for computer access. See http://www.mediawiki.org/wiki/API Yes, I am aware of this as well. Does anyone know of a python class for easily interacting with it, or do I need to roll my own. http://pypi.python.org/pypi?%3Aaction=searchterm=wikipedia ? Thorsten -- http://mail.python.org/mailman/listinfo/python-list
Re: Scraping Wikipedia with Python
maybe you want dbpedia. I did not know about this. Thanks! That is the reason why I ask. This list has an unbelievable collective knowledge and I am certain that asking how much is 2+2 would net an insightful answer that would teach me something. Thank you, Paul, and thank you to the entire Python list! -- Dotan Cohen http://what-is-what.com http://gibberish.co.il -- http://mail.python.org/mailman/listinfo/python-list
Re: Scraping Wikipedia with Python
http://pypi.python.org/pypi?%3Aaction=searchterm=wikipedia ? Thanks, Thorsten, I will go through those. I did not know about that resource, I am not a regular coder. One more resource to add to the toolbox! -- Dotan Cohen http://what-is-what.com http://gibberish.co.il -- http://mail.python.org/mailman/listinfo/python-list
Re: Scraping Wikipedia with Python
Dotan Cohen dotanco...@gmail.com writes: maybe you want dbpedia. I did not know about this. Thanks! You might also like freebase/metaweb. -- http://mail.python.org/mailman/listinfo/python-list
Scraping Wikipedia with Python
I plan on making a geography-learning Anki [1] deck, and Wikipedia has the information that I need in nicely formatted tables on the side of each country's page. Has someone already invented a wheel to parse and store that data (scrape)? It is probably not difficult to code, and within the Wikipedia license, but if that wheel has already been invented then I don't want to redo it. I tried googling for a Wikipedia-specific solution but found none, is there a general purpose solution that I could use? Note that I am a regular Wikipedia contributor and plan on staying within the realm of Wikipedia's rules. [1] http://ichi2.net/anki/ -- Dotan Cohen http://what-is-what.com http://gibberish.co.il -- http://mail.python.org/mailman/listinfo/python-list
Re: Scraping Wikipedia with Python
Dotan Cohen wrote: I plan on making a geography-learning Anki [1] deck, and Wikipedia has the information that I need in nicely formatted tables on the side of each country's page. Has someone already invented a wheel to parse and store that data (scrape)? Wikipedia has an API for computer access. See http://www.mediawiki.org/wiki/API John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: Scraping Wikipedia with Python
Wikipedia has an API for computer access. See http://www.mediawiki.org/wiki/API Yes, I am aware of this as well. Does anyone know of a python class for easily interacting with it, or do I need to roll my own. -- Dotan Cohen http://what-is-what.com http://gibberish.co.il -- http://mail.python.org/mailman/listinfo/python-list
Re: Scraping Wikipedia with Python
On Tue, 11 Aug 2009 21:29:40 +0300, Dotan Cohen wrote: Wikipedia has an API for computer access. See http://www.mediawiki.org/wiki/API Yes, I am aware of this as well. Does anyone know of a python class for easily interacting with it, or do I need to roll my own. Try reading a little there! Starting there I went to http://en.wikipedia.org/wiki/Wikipedia:Creating_a_bot where I found a section on existing bots, comments on how the scraping is not what you want, and even a Python section with a link to something labelled PyWikipediaBot... -- http://mail.python.org/mailman/listinfo/python-list
Re: Scraping Wikipedia with Python
On Tue, Aug 11, 2009 at 12:29 PM, Dotan Cohen dotanco...@gmail.com wrote: Wikipedia has an API for computer access. See http://www.mediawiki.org/wiki/API Yes, I am aware of this as well. Does anyone know of a python class for easily interacting with it, or do I need to roll my own. -- Dotan Cohen https://mwclient.svn.sourceforge.net/svnroot/mwclient/trunk/mwclient/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Scraping Wikipedia with Python
Dotan Cohen dotanco...@gmail.com writes: Thanks. I read the first bit of that page, but did not finish it. Grepping it for Python led to to what I need. maybe you want dbpedia. -- http://mail.python.org/mailman/listinfo/python-list
Re: wikipedia with python
On Mon, 22 Jun 2009 19:23:59 +0200, Andre Engels andreeng...@gmail.com wrote: On Mon, Jun 22, 2009 at 6:58 PM, zelegolaszelego...@gmail.com wrote: Let me know if it's the right place to ask. I'm looking for wiki writen with python where I can import all wikipedia site. If you have any links please let me know. I don't think that's possible. If you wnat to import Wikipedia in a wiki, it will probably have to be MediaWiki - and that's written in PHP. What do you want to use the material for? Well sorry I was not clear. I have a wiki running with mediawiki and I want to import in a wiki written with python. -- http://mail.python.org/mailman/listinfo/python-list
Re: wikipedia with python
On 2009-06-22, ZeLegolas zelego...@gmail.com wrote: On Mon, 22 Jun 2009 19:23:59 +0200, Andre Engels andreeng...@gmail.com wrote: On Mon, Jun 22, 2009 at 6:58 PM, zelegolaszelego...@gmail.com wrote: I'm looking for wiki writen with python where I can import all wikipedia site. PHP. What do you want to use the material for? Well sorry I was not clear. I have a wiki running with mediawiki and I want to import in a wiki written with python. To clarify, check one: [ ] A. You already have mediawiki wiki running on a web server; but, you would prefer to have your wiki power by Python. [ ] B. You already have mediawiki running on a web server and you have another wiki powered by Python; and, you would like to take the information from the python powered wiki and copy it to the mediawiki. [ ] C. You already have mediawiki running on a web server and you would also like to use the functionality of a wiki which is written in Python. Somehow, you would like to somehow access some of the Python's functionality and use it from inside of mediawiki. Perhaps making them work side by side. [ ] D. You already have mediawiki running on a webserver; but, you would like to get data from another mediawiki server that you don't have direct database access to; so, you would like to write a script in Python to scrape the data off of the other mediawiki's website. [ ] E. You already have a wiki running in python on a webserver and you would like to import data from another wiki that is powered by mediawiki. You might or might not have direct access to the database used by the wikimedia server. You would like a Python script to convert/scrap the data. [ ] F. Other -- http://mail.python.org/mailman/listinfo/python-list
Re: wikipedia with python
On Wed, Jun 24, 2009 at 9:17 AM, Tim Hariguser...@ilthio.net wrote: [ ] D. You already have mediawiki running on a webserver; but, you would like to get data from another mediawiki server that you don't have direct database access to; so, you would like to write a script in Python to scrape the data off of the other mediawiki's website. In case D (and more general, to anyone who wants to do something to a Mediawiki wiki using Python), you can use the Python Wikipediabot Framework, http://pywikipediabot.sourceforge.net/ -- André Engels, andreeng...@gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: wikipedia with python
ZeLegolas wrote: On Mon, 22 Jun 2009 19:23:59 +0200, Andre Engels andreeng...@gmail.com wrote: On Mon, Jun 22, 2009 at 6:58 PM, zelegolaszelego...@gmail.com wrote: Let me know if it's the right place to ask. I'm looking for wiki writen with python where I can import all wikipedia site. If you have any links please let me know. I don't think that's possible. If you wnat to import Wikipedia in a wiki, it will probably have to be MediaWiki - and that's written in PHP. What do you want to use the material for? Well sorry I was not clear. I have a wiki running with mediawiki and I want to import in a wiki written with python. Is there anything insufficient in mediawiki, that you think could be satisfied with a python-based wiki? MediaWiki is one of the best wiki software around, so any features not found in MediaWiki or its plugins isn't likely to be available in another wiki (unless written in a language I know is considered a feature). -- http://mail.python.org/mailman/listinfo/python-list
wikipedia with python
Let me know if it's the right place to ask. I'm looking for wiki writen with python where I can import all wikipedia site. If you have any links please let me know. Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: wikipedia with python
Does this help: http://www.mediawiki.org/wiki/MoinMoin On Mon, Jun 22, 2009 at 6:58 PM, zelegolaszelego...@gmail.com wrote: Let me know if it's the right place to ask. I'm looking for wiki writen with python where I can import all wikipedia site. If you have any links please let me know. Thanks -- http://mail.python.org/mailman/listinfo/python-list -- http://www.xing.com/profile/Martin_Marcher http://www.linkedin.com/in/martinmarcher You are not free to read this message, by doing so, you have violated my licence and are required to urinate publicly. Thank you. Please avoid sending me Word or PowerPoint attachments. See http://www.gnu.org/philosophy/no-word-attachments.html -- http://mail.python.org/mailman/listinfo/python-list
Re: wikipedia with python
On Mon, Jun 22, 2009 at 6:58 PM, zelegolaszelego...@gmail.com wrote: Let me know if it's the right place to ask. I'm looking for wiki writen with python where I can import all wikipedia site. If you have any links please let me know. I don't think that's possible. If you wnat to import Wikipedia in a wiki, it will probably have to be MediaWiki - and that's written in PHP. What do you want to use the material for? -- André Engels, andreeng...@gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: wikipedia with python
On Mon, 22 Jun 2009 21:01:16 +0200, Andre Engels andreeng...@gmail.com wrote: On Mon, Jun 22, 2009 at 8:24 PM, ZeLegolaszelego...@gmail.com wrote: Well sorry I was not clear. I have a wiki running with mediawiki and I want to import in a wiki written with python. I don't think it will work, but you could try using the Special:Export page. Thanks I will try. :) I don't choose the wiki base on python yet. Do you know one similar to mediawiki or what is the best wiki that you know? -- http://mail.python.org/mailman/listinfo/python-list
Re: wikipedia with python
On Mon, Jun 22, 2009 at 8:24 PM, ZeLegolaszelego...@gmail.com wrote: Well sorry I was not clear. I have a wiki running with mediawiki and I want to import in a wiki written with python. I don't think it will work, but you could try using the Special:Export page. -- André Engels, andreeng...@gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: wikipedia with python
Andre Engels andreeng...@gmail.com wrote: On Mon, Jun 22, 2009 at 6:58 PM, zelegolaszelego...@gmail.com wrote: Let me know if it's the right place to ask. I'm looking for wiki writen with python where I can import all wikipedia site. If you have any links please let me know. I don't think that's possible. If you wnat to import Wikipedia in a wiki, it will probably have to be MediaWiki - and that's written in PHP. MoinMoin has a MediaWiki format parser (or two). Not 100% compatible, but good enough for some purposes. Templates will be a problem, though. -- --- | Radovan Garabík http://kassiopeia.juls.savba.sk/~garabik/ | | __..--^^^--..__garabik @ kassiopeia.juls.savba.sk | --- Antivirus alert: file .signature infected by signature virus. Hi! I'm a signature virus! Copy me into your signature file to help me spread! -- http://mail.python.org/mailman/listinfo/python-list
