[2.5.1] Script to download and rename bunch of files?

2011-04-08 Thread Gilles Ganault
Hello, Before I go ahead and learn how to write this, I was wondering if someone knew of some source code I could use to download and rename a bunch of files, ie. the equivalent of wget's -O switch? I would provide a two-column list where column 1 would contain the full URL, and column 2

Re: [2.5.1] Script to download and rename bunch of files?

2011-04-08 Thread Gilles Ganault
On Fri, 08 Apr 2011 15:14:27 +0200, Laurent Claessens moky.m...@gmail.com wrote: The following puts in the string `a` the code of the page urlBase : a = urllib.urlopen(urlBase).read() Then you have to write `a` in a file. There could be better way. Thank you. --

Re: [newbie/2.5.1.1] Computing value of a word?

2011-02-17 Thread Gilles Ganault
On Wed, 16 Feb 2011 01:27:06 -0800, Chris Rebert c...@rebertia.com wrote: A = ord('a') - 1 for line in your_file: word = line.strip().lower() score = sum(ord(letter)-A for letter in word) Thanks much Chris. -- http://mail.python.org/mailman/listinfo/python-list

[newbie/2.5.1.1] Computing value of a word?

2011-02-16 Thread Gilles Ganault
Hello, For a game, I need to go through a wordlist, and for each word, compute its value, ie. a=1, b=2, etc. So for instance, NewYork = 14 + 5 + 23 + 25 + 15 + 18 + 11 = 111. Before I write the obvious While loop to go through each line in the input text file, I was wondering if Python didn't

Re: Only one forum app in Python?

2010-07-17 Thread Gilles Ganault
On Sat, 10 Jul 2010 18:37:00 +0200, Bruno Desthuilliers bdesth.quelquech...@free.quelquepart.fr wrote: There are almost a dozen of Python forum apps for Django alone, and Python is known as the language with more web frameworks than keywords. So this list at Wikipedia is out-of-date/wrong, and

Re: Only one forum app in Python?

2010-07-12 Thread Gilles Ganault
On Sat, 10 Jul 2010 18:37:00 +0200, Bruno Desthuilliers bdesth.quelquech...@free.quelquepart.fr wrote: There are almost a dozen of Python forum apps for Django alone, and Python is known as the language with more web frameworks than keywords. Thanks for the tip. I'll head that way. --

Only one forum app in Python?

2010-07-09 Thread Gilles Ganault
Hello I'd like to write a small web app in Python which must include a forum. So I checked the relevant article in Wikipedia, which says that only one forum app is available for Python: http://en.wikipedia.org/wiki/Comparison_of_internet_forum_software_(other) Is Pocoo really the only solution

Re: HTTP server + SQLite?

2010-05-04 Thread Gilles Ganault
On Mon, 3 May 2010 23:07:08 -0700 (PDT), Bryan bryanjugglercryptograp...@yahoo.com wrote: I love SQLite because it solves problems I actually have. For the vast majority of code I write, lite is a good thing, and lite as it is, SQLite can handle several transactions per second. I give SQLite a

HTTP server + SQLite?

2010-05-03 Thread Gilles Ganault
Hello I'd like to build a prototype that will combine a web server as front-end (it must support GZIPping data to the remote client when there are a lot of data to return), and SQLite as back-end, call the server from a VB.Net application, and see how well this works. I want to see if performance

Re: HTTP server + SQLite?

2010-05-03 Thread Gilles Ganault
On Mon, 03 May 2010 11:51:41 +0200, Helmut Jarausch jarau...@igpm.rwth-aachen.de wrote: http://www.karrigell.fr/doc/ Thanks for the tip. -- http://mail.python.org/mailman/listinfo/python-list

Re: Write web apps in Python?

2010-04-15 Thread Gilles Ganault
On Mon, 12 Apr 2010 05:37:54 -0700 (PDT), Luis M. González luis...@gmail.com wrote: You should first investigate the different python web frameworks, choose one and then use the deployment options supported by your choice. These frameworks support several ways to deploy your apps, such as those

Re: Write web apps in Python?

2010-04-15 Thread Gilles Ganault
On Thu, 15 Apr 2010 12:41:56 +0200, Bruno Desthuilliers bruno.42.desthuilli...@websiteburo.invalid wrote: The PHP execution model (mostly based on CGI FWIW) tends to be a bit unpractical for non-trivial applications since you have to rebuild the whole world for each and any incoming request,

Write web apps in Python?

2010-04-12 Thread Gilles Ganault
Hello I'd like to make sure I understand what the options are to write web applications in Python: - à la PHP, using Apache's mod_python - using eg. Lighttpd and configuring it to load the Python interpreter every time a Python script is called (www.jakehilton.com/?q=node/54) - long-running

Re: Write web apps in Python?

2010-04-12 Thread Gilles Ganault
On Mon, 12 Apr 2010 12:13:17 +0200, Daniel Fetchinson fetchin...@googlemail.com wrote: For additional info have a look at http://wiki.python.org/moin/WebProgramming Thanks for the link. -- http://mail.python.org/mailman/listinfo/python-list

[2.5.1/cookielib] How to display specific cookie?

2010-04-07 Thread Gilles Ganault
Hello I'm using ActivePython 2.5.1 and the cookielib package to retrieve web pages. I'd like to display a given cookie from the cookiejar instead of the whole thing: #OK for index, cookie in enumerate(cj): print index, ' : ', cookie #How to display just

Re: [2.5.1.1/dictionary] Change sorting order?

2010-01-24 Thread Gilles Ganault
On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault nos...@nospam.com wrote: To avoid users from creating login names that start with digits in order to be listed at the top, I'd like to sort the list differently every minute so that it'll start with the next letter, eg. display the list from

[2.5.1.1/dictionary] Change sorting order?

2010-01-22 Thread Gilles Ganault
Hello I use a dictionary to keep a list of users connected to a web site. To avoid users from creating login names that start with digits in order to be listed at the top, I'd like to sort the list differently every minute so that it'll start with the next letter, eg. display the list from

Re: [2.5.1.1/dictionary] Change sorting order?

2010-01-22 Thread Gilles Ganault
On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault nos...@nospam.com wrote: I see that dictionaries can be sorted using the... sort() method, but is it possible to have Python start sorting from a different letter? Looks like the solution is to read the list of keys into a list, sort the list

Re: [2.5.1.1/dictionary] Change sorting order?

2010-01-22 Thread Gilles Ganault
On 22 Jan 2010 13:35:26 GMT, Neil Cerutti ne...@norwich.edu wrote: Resorting is more work than is needed. Just choose a different starting index each time you display the names, and set up your lister to wrap-around to your arbitrary starting index. Thanks. In this case, it means that in each

Re: [2.5.1.1/dictionary] Change sorting order?

2010-01-22 Thread Gilles Ganault
On Fri, 22 Jan 2010 14:09:43 +0100, Jean-Michel Pichavant jeanmic...@sequans.com wrote: Sorry, the code I provided produce this output: ['1a', 'a', 'ac', 'av', 'b', 'c'] ['a', 'ac', 'av', 'b', 'c', '1a'] ['b', 'c', '1a', 'a', 'ac', 'av'] ['c', '1a', 'a', 'ac', 'av', 'b'] ['1a', 'a', 'ac', 'av',

Re: [2.5.1.1/dictionary] Change sorting order?

2010-01-22 Thread Gilles Ganault
On Fri, 22 Jan 2010 09:49:32 -0500, Dave Angel da...@ieee.org wrote: Seems to me the other solutions I've seen so far are more complex than needed. I figure you either want an unordered list, in which case you could use random.shuffle(), or you want a list that's sorted, but starts somewhere

Re: [2.5.1.1/dictionary] Change sorting order?

2010-01-22 Thread Gilles Ganault
On 22 Jan 2010 15:24:58 GMT, Duncan Booth duncan.bo...@invalid.invalid wrote: Here's another: Thanks for the sample. It work great, except that it also runs when the header character doesn't match any item in the list: === import bisect connected = [] connected.append(_test)

Re: [2.5.1.1/dictionary] Change sorting order?

2010-01-22 Thread Gilles Ganault
On Fri, 22 Jan 2010 17:21:02 +0100, Jean-Michel Pichavant jeanmic...@sequans.com wrote: Ok I realized that picking up a random index prevent from grouping names starting with the same letter (to ease visual lookup). Then go for the random char, and use char comparison (my first example). Yup, I

[PyWin32] Hello, world?

2009-10-28 Thread Gilles Ganault
Hello I'm reading O'Reily's Python Programming on Win32, but couldn't find a simple example on how to create a window with just a label and pushbutton. If someone has such a basic example handy, I'm interested. Thank you. -- http://mail.python.org/mailman/listinfo/python-list

Concurrent threads to pull web pages?

2009-10-01 Thread Gilles Ganault
Hello I recently asked how to pull companies' ID from an SQLite database, have multiple instances of a Python script download each company's web page from a remote server, eg. www.acme.com/company.php?id=1, and use regexes to extract some information from each page. I need to run

Re: [SQL] Pick random rows from SELECT?

2009-09-23 Thread Gilles Ganault
On Mon, 21 Sep 2009 21:40:02 -0700, Dennis Lee Bieber wlfr...@ix.netcom.com wrote: I'd suggest either a pool of threads -- 5-10, each reading company names from a shared QUEUE, which is populated by the main thread (remember to commit() so that you don't block on database updates by the

[SQL] Pick random rows from SELECT?

2009-09-21 Thread Gilles Ganault
Hello I have a working Python script that SELECTs rows from a database to fetch a company's name from a web-based database. Since this list is quite big and the site is the bottleneck, I'd like to run multiple instances of this script, and figured a solution would be to pick rows at

Re: Python for professsional Windows GUI apps?

2009-08-25 Thread Gilles Ganault
On Tue, 25 Aug 2009 13:24:39 +0200, Wolfgang Keller felip...@gmx.net wrote: The area of _desktop_ database application development indeed looks like a vast and very hostile desert in the Python landscape. The only framework that seems to be worth trying is Dabo. Unfortunately there's little

Python for professsional Windows GUI apps?

2009-08-24 Thread Gilles Ganault
Hello I was wondering if some people in this ng use Python and some GUI toolkit (PyWin32, wxWidgets, QT, etc.) to build professional applications, and if yes, what it's like, the pros and cons, etc. I'm especially concerned about the lack of controls, the lack of updates (lots of

Re: Converting DD MM YYYY into YYYY-MM-DD?

2009-08-18 Thread Gilles Ganault
Thanks everyone for the help. This script is just a one-shot thingie on my work host, not as a web script or anything professional. On Mon, 17 Aug 2009 17:05:28 -0700 (PDT), Jonathan Gardner jgard...@jonathangardner.net wrote: Unfortunately, there isn't any string to date parsers in the built-

Re: Converting DD MM YYYY into YYYY-MM-DD?

2009-08-18 Thread Gilles Ganault
On Tue, 18 Aug 2009 17:10:50 +1000, Ben Finney ben+pyt...@benfinney.id.au wrote: Luckily, you have access to the documentation to find out. I never used groups before. Thanks for showing me. At this point, the script is almost done, but the regex fails if the month contains accented characters

Re: Converting DD MM YYYY into YYYY-MM-DD?

2009-08-18 Thread Gilles Ganault
On Tue, 18 Aug 2009 01:11:20 -0700, Rami Chowdhury rami.chowdh...@gmail.com wrote: Could you let me know which platform this is on (Windows, *nix)? It may be a locale encoding issue -- the locale.setlocale() function allows the second argument to be a tuple of (locale_code, encoding), as below:

Re: Converting DD MM YYYY into YYYY-MM-DD?

2009-08-18 Thread Gilles Ganault
On Tue, 18 Aug 2009 10:52:41 +0200, Gilles Ganault nos...@nospam.com wrote: I find it odd that the regex library can't handle European characters :-/ Ha, found it! :-) http://www.regular-expressions.info/python.html = # -*- coding: latin-1 -*- import locale import re locale.setlocale

Re: Converting DD MM YYYY into YYYY-MM-DD?

2009-08-18 Thread Gilles Ganault
On Tue, 18 Aug 2009 20:03:47 +1000, Ben Finney ben+pyt...@benfinney.id.au wrote: The principles of handling text in Python: Get it to internal Unicode objects as soon as possible, handle it as Unicode for as long as possible, and only encode it to some byte stream for output as late as possible.

Converting DD MM YYYY into YYYY-MM-DD?

2009-08-17 Thread Gilles Ganault
Hello, I need to convert DD MM dates into the MySQL-friendly -MM-DD, and translate the month name from literal French to its numeric equivalent (eg. Janvier into 01). Here's an example: SELECT dateinscription, dateconnexion FROM membres LIMIT 1; 26 Mai 2007|17 Août 2009 - 09h20

Compiling Python for uCLinux appliance?

2009-07-30 Thread Gilles Ganault
Hello I just got a small appliance based on a Blackfin CPU with 64MB RAM and 258MB NAND flash. Using the stock software, there's about 30MB of RAM left. Besides C/C++ and shel scripts, I was wondering if it were realistic to upload a few Python scripts in such a small appliance? Thank you. --

Are Python-based web frameworks reliable enough?

2009-05-25 Thread Gilles Ganault
Hello Until now, the modest web apps I wrote were all in PHP because it's available on just about any hosted server. I now have a couple of ideas for applications where I would deploy my own servers, so that I'd rather write them in Python because I find the language more pleasant to

Something weird about re.finditer()

2009-04-15 Thread Gilles Ganault
Hello I stumbled upon something funny while downloading web pages and trying to extract one or more blocks from a page: Even though Python seems to return at least one block, it doesn't actually enter the for loop: == re_block = re.compile('before (.+?) after',re.I|re.S|re.M) #Here,

Why does Python show the whole array?

2009-04-08 Thread Gilles Ganault
Hello I'd like to go through a list of e-mail addresses, and extract those that belong to well-known ISP's. For some reason I can't figure out, Python shows the whole list instead of just e-mails that match: === script test = t...@gmail.com isp = [gmail.com, yahoo.com] for item in isp:

Re: Why does Python show the whole array?

2009-04-08 Thread Gilles Ganault
On Wed, 08 Apr 2009 12:11:55 +0200, Ulrich Eckhardt eckha...@satorlaser.com wrote: find() returns the index where it is found or -1 if it is not found. Both an index0 or a -1 evaluate to True when used as conditional expression. Thanks everyone. I shouldn't have assumed that if test.find(item):

Re: [regex] How to check for non-space character?

2009-03-22 Thread Gilles Ganault
On Sat, 21 Mar 2009 08:53:10 -0500, Tim Chase python.l...@tim.thechases.com wrote: It looks like it's these periods that are throwing you off. Just remove them. For a 3rd syntax: (\S)(\d{5}) the \S (capital, instead of \s) is any NON-white-space character Thanks guys for the tips. --

[regex] How to check for non-space character?

2009-03-21 Thread Gilles Ganault
Hello Some of the adresses are missing a space between the streetname and the ZIP code, eg. 123 Main Street01159 Someville The following regex doesn't seem to work: #Check for any non-space before a five-digit number re_bad_address = re.compile('([^\s].)(\d{5}) ',re.I | re.S | re.M) I also

Re: Encoding/decoding: Still don't get it :-/

2009-03-16 Thread Gilles Ganault
On Fri, 13 Mar 2009 14:24:52 +0100, Peter Otten __pete...@web.de wrote: It seems the database gives you the strings as unicode. When a unicode string is printed python tries to encode it using sys.stdout.encoding before writing it to stdout. As you run your script on the windows commmand line that

[ActivePython 2.5.1.1] Why does Python not return first line?

2009-03-15 Thread Gilles Ganault
Hello I'm stuck at why Python doesn't return the first line in this simple regex: === response = spanAddress :/span/td\r\t\ttd\r\t\t\t3 Abbey Road, St Johns Wood br /\r\t\t\tLondon, NW8 9AY\t\t/td re_address = re.compile('spanAddress :/span/td.+?td(.+?)/td',re.I | re.S | re.M) address

Re: [ActivePython 2.5.1.1] Why does Python not return first line?

2009-03-15 Thread Gilles Ganault
On Mon, 16 Mar 2009 01:14:00 +0100, Gilles Ganault nos...@nospam.com wrote: I'm stuck at why Python doesn't return the first line in this simple regex Found it: Python does extract the token, but displaying it requires removing hidden chars: = response = spanAddress :/span/td\r\t\ttd\r\t\t

Encoding/decoding: Still don't get it :-/

2009-03-13 Thread Gilles Ganault
Hello I must be dense, but I still don't understand 1) why Python sometimes barfs out this type of error when displaying text that might not be Unicode-encoded, 2) whether I should use encode() or decode() to solve the issue, or even 3) if this is a Python issue or due to APWS SQLite wrapper that

[Web 2.0] Added-value of frameworks?

2009-02-04 Thread Gilles Ganault
Hello If I wanted to build some social web site such as Facebook, what do frameworks like Django or TurboGears provide over writing a site from scratch using Python? Thank you for your feedback. -- http://mail.python.org/mailman/listinfo/python-list

Re: [2.5.1] Comparing dates?

2009-02-04 Thread Gilles Ganault
On Mon, 2 Feb 2009 22:00:53 +0100, Martin mar...@marcher.name wrote: as suggested, the DBA should seriously think about defining the correct type of the column here, for intermediate use and getting stuff to work you could use a view and define some stored procedures on it so that inserting

[2.5.1] Comparing dates?

2009-02-02 Thread Gilles Ganault
Hello I have data in an SQL database where one column contains a date formated as DD/MM/Y. I need to select all rows where the date is before, say Feb 1st 2009, ie. 01/02/2009. Is there a command in Python that does this easily, or should I look into whatever date() function the SQL

Re: [2.5.1] Comparing dates?

2009-02-02 Thread Gilles Ganault
On Mon, 02 Feb 2009 20:06:02 +1100, Ben Finney bignose+hates-s...@benfinney.id.au wrote: The Python data types for date and time are in the ‘datetime’ module URL:http://www.python.org/doc/2.6/library/datetime. Create a ‘datetime’ object for each value you want, then compare them. Thanks guys. For

[urllib2] 302 - can't get cookie

2009-01-07 Thread Gilles Ganault
Hello I'm using urllib2 to connect to a web server with POST, and then the server sends a cookie to hold the session ID, but also redirects the user to another page: === HTTP/1.1 302 Found Date: Wed, 07 Jan 2009 11:20:51 GMT Server: Apache Set-Cookie: PHPSESSID=4015f14eb04dc81159253a9533a7c590;

[http] Example with proxy + post + cookie?

2009-01-06 Thread Gilles Ganault
Hello I need to write a script that goes out through a proxy, connects with a POST query, and receives a cookie for a Session ID. I didn't find an example on the Net that did it all three, but only some of the features. Does someone have one handy by any chance? Thank you. --

Re: [email/quoprimime.py] AttributeError: 'tuple' object has no attribute 'lstrip'

2009-01-06 Thread Gilles Ganault
On Mon, 05 Jan 2009 17:54:19 -0500, Terry Reedy tjre...@udel.edu wrote: for email in rows: To = email Thanks guys. Turns out email is a tuple, so here's how to extract the columns: for email in rows: email=email[0] -- http://mail.python.org/mailman/listinfo/python-list

[email/quoprimime.py] AttributeError: 'tuple' object has no attribute 'lstrip'

2009-01-04 Thread Gilles Ganault
Hello I successfully use the email package to send e-mail from Python scripts, but this script fails when I fetch addresses from an SQLite database where data is Unicode-encoded: == from email.MIMEText import MIMEText import smtplib,sys import apsw

[2.5.1] Str.Replace() doesn't work?

2008-12-26 Thread Gilles Ganault
Hello I need to parse an HTML file where records aren't homogenous, so I figured I could run a first loop to add an unused character at the beginning of each record, and then run a second loop to actually parse each record. I can't figure out why the script is not returning anything in

Re: [2.5.1] Str.Replace() doesn't work?

2008-12-26 Thread Gilles Ganault
On Fri, 26 Dec 2008 15:16:49 +0100, Gilles Ganault nos...@nospam.com wrote: I can't figure out why the script is not returning anything in the for m in matches block: Pfff, found it 5mn after posting ;-) s/reponse =/response =/ Sorry guys. -- http://mail.python.org/mailman/listinfo/python-list

[SQL] Right way to set a variable to NULL?

2008-12-26 Thread Gilles Ganault
Hello I use regexes to extract information from a text file. Some of the records don't have e-mails or www addresses, so those must match Null in SQL, but None doesn't work as expected: === if itemmatch: web = itemmatch.group(1).strip() else:

Are Django/Turbogears too specific?

2008-12-21 Thread Gilles Ganault
Hi I'd like to rewrite a Web 2.0 PHP application in Python with AJAX, and it seems like Django and Turbogears are the frameworks that have the most momentum. I'd like to use this opportunity to lower the load on servers, as the PHP application wasn't built to fit the number of users hammering

[cookielib] How to add cookies myself?

2008-12-16 Thread Gilles Ganault
Hello I'm using urllib and urlib to download data from a web server that requires cookies. The issue I'm having, is the server uses JavaScript in the response to insert new cookies and send them with the next query, so I need to manually add a couple of cookies in the CookieJar, but I don't know

Re: [2.5.1] ShiftJIS to Unicode?

2008-11-27 Thread Gilles Ganault
On Thu, 27 Nov 2008 01:00:28 +, MRAB [EMAIL PROTECTED] wrote: No problem here: import urllib data = urllib.urlopen(http://www.amazon.co.jp/;).read() decoded_data = data.decode(shift-jis) Thanks, but it seems like some pages contain ShiftJIS mixed with some other code page, and Python

[2.5.1] ShiftJIS to Unicode?

2008-11-26 Thread Gilles Ganault
Hello I'm trying to read pages from Amazon JP, whose web pages are supposed to be encoded in ShiftJIS, and decode contents into Unicode to keep Python happy: www.amazon.co.jp meta http-equiv=content-type content=text/html; charset=Shift_JIS / But this doesn't work: == m =

Using dictionary to hold regex patterns?

2008-11-23 Thread Gilles Ganault
Hello After downloading a web page, I need to search for several patterns, and if found, extract information and put them into a database. To avoid a bunch of if m, I figured maybe I could use a dictionary to hold the patterns, and loop through it: == pattern = {} pattern[pattern1] =

Re: Using dictionary to hold regex patterns?

2008-11-23 Thread Gilles Ganault
On Sun, 23 Nov 2008 17:55:48 +, Arnaud Delobelle [EMAIL PROTECTED] wrote: But there is no reason why you should use a dictionary; just use a list of key-value pairs: patterns = [ (pattern1, re.compile(.+?/td.+?(.+?)/td), Thanks for the tip, but... I thought that lists could only use

Re: Using dictionary to hold regex patterns?

2008-11-23 Thread Gilles Ganault
On Sun, 23 Nov 2008 17:55:48 +, Arnaud Delobelle [EMAIL PROTECTED] wrote: But there is no reason why you should use a dictionary; just use a list of key-value pairs: Thanks for the tip. I didn't know it was possible to use arrays to hold more than one value. Actually, it's a better solution,

Re: Using dictionary to hold regex patterns?

2008-11-23 Thread Gilles Ganault
On Sun, 23 Nov 2008 23:18:06 +, MRAB [EMAIL PROTECTED] wrote: A list is an ordered collection of items. Each item can be anything: a string, an integer, a dictionary, a tuple, a list... Yup, learned something new today. Naively, I though a list was index=value, where value=a single piece of

More elegant way to try running a function X times?

2008-11-19 Thread Gilles Ganault
Hello As a newbie, it's pretty likely that there's a smarter way to do this, so I'd like to check with the experts: I need to try calling a function 5 times. If successful, move on; If not, print an error message, and exit the program: = success = None for i in range(5): #Try to

Re: More elegant way to try running a function X times?

2008-11-19 Thread Gilles Ganault
On 19 Nov 2008 14:37:06 + (GMT), Sion Arrowsmith [EMAIL PROTECTED] wrote: Note very carefully that the else goes with the for and not the if. Thanks guys. -- http://mail.python.org/mailman/listinfo/python-list

[urllib2] No time-out?

2008-11-16 Thread Gilles Ganault
Hello I'm using urllib2 to download web pages. The strange thing in the code below, is that it seems like urllib2.urlopen retries indefinitely by itself instead of raising an exception: = timeout = 30 socket.setdefaulttimeout(timeout) i = 0 while i 5:

Re: [UnicodeEncodeError] Don't know what else to try

2008-11-15 Thread Gilles Ganault
On Fri, 14 Nov 2008 17:39:00 +0100, Martin v. Löwis [EMAIL PROTECTED] wrote: Can you first please report what happened when you add the print statement? Thanks guys, I found how to handle this: === for id in rows: #Says Unicode, but it's actually not #print type(id[1])

Comparing value in two dictionaries?

2008-11-15 Thread Gilles Ganault
Hello I fill two dictionaries with the same number of keys, and then need to compare the value for each key, eg. #Pour chaque APE, comparaison societe.ape.nombre et verif.ape.nombre import apsw # dic1={} [...] rows=list(cursor.execute(sql)) for id in rows: dic1[id[0]] =

[UnicodeEncodeError] Don't know what else to try

2008-11-14 Thread Gilles Ganault
Hello Data that I download from the web seems to be using different code pages at times, and Python doesn't like this. Google returned a way to handle this, but I'm still getting an error: print output.decode('utf-8') File C:\Python25\lib\encodings\utf_8.py, line 16, in decode

Re: [UnicodeEncodeError] Don't know what else to try

2008-11-14 Thread Gilles Ganault
On Fri, 14 Nov 2008 11:01:27 +0100, Martin v. Löwis [EMAIL PROTECTED] wrote: Add print type(output) here. If it says unicode, reconsider the next line print output.decode('utf-8') In case the string fetched from a web page turns out not to be Unicode

Proxy + cookie: urllib2? httplib2? Other?

2008-11-14 Thread Gilles Ganault
Hello I need a library that supports both going out through a proxy, and handling cookies automagically (the server uses a sessionID to keep track of the user). UrlLib2 supports the proxy part, httplib2 supports the cookie part but... Google didn't return code that shows both uses in the same

Re: Proxy + cookie: urllib2? httplib2? Other?

2008-11-14 Thread Gilles Ganault
On Fri, 14 Nov 2008 16:03:13 +0100, Gilles Ganault [EMAIL PROTECTED] wrote: I need a library that supports both going out through a proxy, and handling cookies automagically (the server uses a sessionID to keep track of the user). For those interested, it seems like a good combination is urllib2

[Regex] Search and replace?

2008-11-13 Thread Gilles Ganault
Hello I need to iterate through a variable, and for each pattern that matches, replace this with something else. I read the chapter in www.amk.ca/python/howto/regex/, but the output is wrong: === #Extract two bits, and rewrite the HTML person = re.compile('tr onMouseOver=(?Pitem1.+?).+?a

Re: Does Python mess with CRLFs?

2008-11-12 Thread Gilles Ganault
On Wed, 12 Nov 2008 12:04:07 +0100, Gilles Ganault [EMAIL PROTECTED] wrote: I wonder if Python rewrites CRLFs when reading a text file with open/read? For those seeing the same thing, the answer is yes: On Windows, the code above turns CRLF into LF. I tried rb instead of r, with no difference

Does Python mess with CRLFs?

2008-11-12 Thread Gilles Ganault
Hello I'm stuck at understanding why Python can't extract some bit from an HTML file using regexes, although I can find it just fine with UltraEdit. I wonder if Python rewrites CRLFs when reading a text file with open/read? Here's the code: == f = open(content.html, r) content =

[Array] Finding index that contains part of a string?

2008-11-12 Thread Gilles Ganault
Hello Out of curiosity, is there a better way in Python to iterate through an array, and return the index of each item that contains the bit somewhere in its value, ie. index() doesn't work because it only returns if the value only contains the item I'm looking for. This works: next =

Re: [Newbie] Strange output from list

2008-11-11 Thread Gilles Ganault
On Mon, 10 Nov 2008 20:02:39 -0600, Andrew [EMAIL PROTECTED] wrote: sql = 'SELECT id FROM master' rows=list(cursor.execute(sql)) for id in rows: sql = 'SELECT COUNT(code) FROM companies WHERE code=%s' % id[0] result = list(cursor.execute(sql)) print Code=%s, number=%s %

[Newbie] Strange output from list

2008-11-10 Thread Gilles Ganault
Hello I'm getting some unwanted result when SELECTing data from an SQLite database: == sql = 'SELECT id FROM master' rows=list(cursor.execute(sql)) for id in rows: sql = 'SELECT COUNT(code) FROM companies WHERE code=%s' % id[0] result = list(cursor.execute(sql)) print

[urllib2 + Tor] How to handle 404?

2008-11-07 Thread Gilles Ganault
Hello I'm using the urllib2 module and Tor as a proxy to download data from the web. Occasionnally, urlllib2 returns 404, probably because of some issue with the Tor network. This code doesn't solve the issue, as it just loops through the same error indefinitely: = for id in rows:

Re: [2.5.1] Building loop with some exceptions?

2008-11-07 Thread Gilles Ganault
On 04 Nov 2008 22:34:49 GMT, Steven D'Aprano [EMAIL PROTECTED] wrote: for i in range(1, 100): if i in (4, 34, 40, 44, 48, 54, 57, 67, 76, 83, 89): continue do_rest_of_processing Thanks for the sample. -- http://mail.python.org/mailman/listinfo/python-list

Re: Building loop with some exceptions?

2008-11-07 Thread Gilles Ganault
On Tue, 4 Nov 2008 15:55:06 -0800 (PST), alex23 [EMAIL PROTECTED] wrote: exclusions = [04, 34, 40, 44, 48, 54, 57, 67, 76, 83, 89] for i in (x for x in xrange(1,100) if x not in exclusions): Thanks guys. -- http://mail.python.org/mailman/listinfo/python-list

[2.5.1] Building loop with some exceptions?

2008-11-04 Thread Gilles Ganault
Hello I need to call a URL through a loop that starts at 01 and ends at 99, but some of the steps must be ignored: = url = http://www.acme.com/list?code=; p = re.compile(^(\d+)\t(.+)$) for i=01 to 99 except 04, 34, 40, 44, 48, 54, 57, 67, 76, 83, 89: f = urllib.urlopen(url + i)

Re: Building loop with some exceptions?

2008-11-04 Thread Gilles Ganault
On Tue, 4 Nov 2008 11:22:27 -0800 (PST), Aaron Brady [EMAIL PROTECTED] wrote: for i=01 to 99 except 04, 34, 40, 44, 48, 54, 57, 67, 76, 83, 89: sorted( list( set( domain ) - set( exceptions ) ) ) Set subtraction. Thanks a lot but... I don't know what the above means :-/ --

Re: Building loop with some exceptions?

2008-11-04 Thread Gilles Ganault
On Tue, 4 Nov 2008 12:10:28 -0800 (PST), Matimus [EMAIL PROTECTED] wrote: I would just do something like this (not tested): Thanks a lot guys :-) Worked first time. I just have the usual issue with ASCII/Unicode: === cursor.execute(sql) UnicodeDecodeError: 'ascii' codec can't decode

[2.5.1] UnicodeDecodeError: 'ascii' codec can't decode byte?

2008-10-29 Thread Gilles Ganault
Hello I'm getting this error while downloading and parsing web pages: = title = m.group(1) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 48: ordinal not in range(128) = From what I understand, it's because some strings are Unicode, and hence contain

[newbie] Right way to access item in array?

2008-10-28 Thread Gilles Ganault
Hello I'd like to know what the right way is to access an item in a row as returned by a database: = import apsw connection=apsw.Connection(test.sqlite) cursor=connection.cursor() rows=cursor.execute(SELECT isbn,price FROM books WHERE price IS NULL) for row in rows: #Is this

Re: Urllib vs. FireFox

2008-10-28 Thread Gilles Ganault
On Fri, 24 Oct 2008 13:15:49 -0700 (PDT), Mike Driscoll [EMAIL PROTECTED] wrote: On Oct 24, 2:53 pm, Rex [EMAIL PROTECTED] wrote: By the way, if you're doing non-trivial web scraping, the mechanize module might make your work much easier. You can install it with

Re: [newbie] Right way to access item in array?

2008-10-28 Thread Gilles Ganault
On Tue, 28 Oct 2008 12:12:26 +0100, Gerhard Häring [EMAIL PROTECTED] wrote: You can do it even in one step with APSW (and pysqlite, and others): for isbn, price in cur.execute(select isbn, price ...): Thanks much guys. For those interested, here's some working code: == import apsw

Re: [newbie] Right way to access item in array?

2008-10-28 Thread Gilles Ganault
On Tue, 28 Oct 2008 09:56:11 -0400, Steve Holden [EMAIL PROTECTED] wrote: If you are dealing with a DB API-compliant module then the return value from the cursor's execute method is undefined, and you need to call one of the fetch methods to extract the retrieved data. Thanks for pointing it out.

URL as input - IOError: [Errno 2] The system cannot find the path specified

2008-10-24 Thread Gilles Ganault
Hello I'm trying to use urllib to download web pages with the GET method, but Python 2.5.1 on Windows turns the URL into something funny: url = amazon.fr/search/index.php?url=search [...] IOError: [Errno 2] The system cannot find the path specified:

Re: URL as input - IOError: [Errno 2] The system cannot find the path specified

2008-10-24 Thread Gilles Ganault
On 24 Oct 2008 18:02:45 GMT, Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote: This URL lacks the protocol! Correct would be http://amazon.fr… (I guess). Thanks, that did it :) -- http://mail.python.org/mailman/listinfo/python-list

Urllib vs. FireFox

2008-10-24 Thread Gilles Ganault
Hello After scratching my head as to why I failed finding data from a web using the re module, I discovered that a web page as downloaded by urllib doesn't match what is displayed when viewing the source page in FireFox. For instance, when searching Amazon for Wargames: URLLIB: a

[2.5.1] Converting string to int?

2008-10-23 Thread Gilles Ganault
Hello I'm using the APSW wrapper to SQLite, and I'm stuck at how to pass data from a dictionary to the database which expects an integer: #array filled by reading a two-column text file as input for (isbn,carton) in data.items(): #TypeError: int argument required sql = INSERT

Re: [APSW] SELECT COUNT(*) not succesfull?

2008-10-23 Thread Gilles Ganault
On Thu, 23 Oct 2008 00:24:01 -0200, Gabriel Genellina [EMAIL PROTECTED] wrote: In case you didn't notice, B.D. already provided the answer you're after - reread his 3rd paragraph from the end. Yes, but it doesn't work with this wrapper (APSW version 3.5.9-r1): The recommended way is to pass

Re: [2.5.1] Converting string to int?

2008-10-23 Thread Gilles Ganault
On Thu, 23 Oct 2008 09:19:07 +0200, Gilles Ganault [EMAIL PROTECTED] wrote: I'm using the APSW wrapper to SQLite, and I'm stuck at how to pass data from a dictionary to the database which expects an integer: Found it: Apparently, this wrapper uses a different placeholder and takes care

[APSW] SELECT COUNT(*) not succesfull?

2008-10-22 Thread Gilles Ganault
Hello I'm trying to use the APSW package to access a SQLite database, but can't find how to check if a row exists. I just to read a tab-separated file, extract a key/value from each line, run SELECT COUNT(*) to check whether this tuple exists in the SQLite database, and if not, run an INSERT.

Re: [APSW] SELECT COUNT(*) not succesfull?

2008-10-22 Thread Gilles Ganault
On Wed, 22 Oct 2008 18:35:35 +0200, Bruno Desthuilliers [EMAIL PROTECTED] wrote: It is - the problem is that cursor.execute doesn't return what you think... Truth is that according to the db-api specification, the return value of cursor.execute is not defined (IOW : can be absolutely anything).

Re: UnicodeDecodeError: 'ascii' codec can't decode byte

2008-06-17 Thread Gilles Ganault
On Tue, 17 Jun 2008 09:23:28 +0200, Peter Otten [EMAIL PROTECTED] wrote: Assuming that encoding is UTF-8 and that apsw can cope with unicode, try to convert your data to unicode before feeding it to the database api: sql = INSERT INTO mytable (col1,col2) VALUES (?,?) rows =

  1   2   >