Re: UnicodeDecodeError issue

2013-09-04 Thread Steven D'Aprano
On Thu, 05 Sep 2013 13:59:17 +1000, Chris Angelico wrote: > On Thu, Sep 5, 2013 at 1:07 PM, Steven D'Aprano > wrote: >> Technically, it's not ASCII, since ASCII only knows about bytes \x00 >> through \x7F (decimal 0 through 127). That's why it isn't correct to >> describe Python bytes strings as

Re: UnicodeDecodeError issue

2013-09-04 Thread Cameron Simpson
On 04Sep2013 12:38, Dave Angel wrote: | 'file' isn't magic. Chuckle... | What heuristics it uses, I don't know, but it has | hundreds of them. ... because the file of heuristics is /usr/share/file/magic :-) See "man 5 magic". Cheers, -- Cameron Simpson A vacuum is a hell of a lot better tha

Re: UnicodeDecodeError issue

2013-09-04 Thread Chris Angelico
On Thu, Sep 5, 2013 at 1:07 PM, Steven D'Aprano wrote: > Technically, it's not ASCII, since ASCII only knows about bytes \x00 > through \x7F (decimal 0 through 127). That's why it isn't correct to > describe Python bytes strings as "ASCII strings". They're byte strings > that happen to be displaye

Re: UnicodeDecodeError issue

2013-09-04 Thread Steven D'Aprano
On Thu, 05 Sep 2013 00:17:36 +, Dave Angel wrote: > On 4/9/2013 10:29, Ferrous Cranus wrote: > >> Στις 4/9/2013 3:38 μμ, ο/η Dave Angel έγραψε: >>> 'file' isn't magic. And again, it doesn't look at the filename, it >>> looks at the content. >> So, you are saying that it looks a the content o

Re: UnicodeDecodeError issue

2013-09-04 Thread Dave Angel
On 4/9/2013 10:29, Ferrous Cranus wrote: > Στις 4/9/2013 3:38 μμ, ο/η Dave Angel έγραψε: >> 'file' isn't magic. And again, it doesn't look at the filename, it >> looks at the content. > So, you are saying that it looks a the content of the file and not of > what encoding we used to save the file

Re: UnicodeDecodeError issue

2013-09-04 Thread Ferrous Cranus
Στις 4/9/2013 3:38 μμ, ο/η Dave Angel έγραψε: 'file' isn't magic. And again, it doesn't look at the filename, it looks at the content. So, you are saying that it looks a the content of the file and not of what encoding we used to save the file into? But the contents have within: f.write(b'\x

Re: UnicodeDecodeError issue

2013-09-04 Thread wxjmfauth
Le mercredi 4 septembre 2013 10:01:50 UTC+2, Antoon Pardon a écrit : > Op 03-09-13 17:23, wxjmfa...@gmail.com schreef: > > > > > > > > > > > The Latin alphabet uses Greek lettering. > > > > > > The Cyrillic alphabet uses Greek lettering. > > > > > > Greek: One should not confus

Re: UnicodeDecodeError issue

2013-09-04 Thread Dave Angel
On 4/9/2013 07:38, Ferrous Cranus wrote: > Στις 4/9/2013 2:26 μμ, ο/η Dave Angel έγραψε: >> So first in the interpreter, I ran f = open("junk.txt", "w") f.write(b'\xb6\xe3\xed\xf9\xf3\xf4\xef\xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe

Re: UnicodeDecodeError issue

2013-09-04 Thread Ferrous Cranus
Στις 4/9/2013 2:26 μμ, ο/η Dave Angel έγραψε: On 4/9/2013 04:35, Ferrous Cranus wrote: Τη Δευτέρα, 2 Σεπτεμβρίου 2013 9:28:36 μ.μ. UTC+3, ο χρήστης Dave Angel έγραψε: On 2/9/2013 11:05, Ferrous Cranus wrote: Στις 2/9/2013 3:21 μμ, ο/η Dave Angel έγραψε: Starting with the byte string in

Re: UnicodeDecodeError issue

2013-09-04 Thread Dave Angel
On 4/9/2013 04:35, Ferrous Cranus wrote: > Τη Δευτέρα, 2 Σεπτεμβρίου 2013 9:28:36 μ.μ. UTC+3, ο χρήστης Dave Angel > έγραψε: >> On 2/9/2013 11:05, Ferrous Cranus wrote: >> >> >> >> > Στις 2/9/2013 3:21 μμ, ο/η Dave Angel έγραψε: >> >> >> Starting with the byte string in the error message: >>

Re: UnicodeDecodeError issue

2013-09-04 Thread Ferrous Cranus
Τη Δευτέρα, 2 Σεπτεμβρίου 2013 9:28:36 μ.μ. UTC+3, ο χρήστης Dave Angel έγραψε: > On 2/9/2013 11:05, Ferrous Cranus wrote: > > > > > Στις 2/9/2013 3:21 μμ, ο/η Dave Angel έγραψε: > > >> Starting with the byte string in the error message: > > > f = open("junk.txt", "w") > > > f.write(b

Re: UnicodeDecodeError issue

2013-09-04 Thread Antoon Pardon
Op 03-09-13 17:23, wxjmfa...@gmail.com schreef: > > > The Latin alphabet uses Greek lettering. > > The Cyrillic alphabet uses Greek lettering. > > Greek: One should not confuse modern Greek > with ancient Greek, polytonic Greek full > of diacritics. > > Plenty of European languages (~

Re: UnicodeDecodeError issue

2013-09-03 Thread wxjmfauth
Le lundi 2 septembre 2013 16:44:34 UTC+2, MRAB a écrit : > On 02/09/2013 13:24, Dave Angel wrote: > > > On 2/9/2013 07:56, MRAB wrote: > > > > > >> On 02/09/2013 12:38, Dave Angel wrote: > > > > > > > > > > > >>> ¶γνωστοόνομα συστήματος > > >>> > > >>> I don't have a clue what it migh

Re: UnicodeDecodeError issue

2013-09-03 Thread Steven D'Aprano
On Tue, 03 Sep 2013 08:45:00 +1000, Chris Angelico wrote: > On Tue, Sep 3, 2013 at 12:44 AM, MRAB > wrote: >> I don't know Greek either, and I don't think there's any other language >> that uses the Greek alphabet. > > Assuming you don't count mathematics as a language. There are a few languag

Re: UnicodeDecodeError issue

2013-09-02 Thread Joel Goldstick
On Mon, Sep 2, 2013 at 6:45 PM, Chris Angelico wrote: > On Tue, Sep 3, 2013 at 12:44 AM, MRAB wrote: >> I don't know Greek either, and I don't think there's any other language >> that uses the Greek alphabet. > > Assuming you don't count mathematics as a language. You need to be rigorous to make

Re: UnicodeDecodeError issue

2013-09-02 Thread Chris Angelico
On Tue, Sep 3, 2013 at 12:44 AM, MRAB wrote: > I don't know Greek either, and I don't think there's any other language > that uses the Greek alphabet. Assuming you don't count mathematics as a language. ChrisA -- http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError issue

2013-09-02 Thread Dave Angel
On 2/9/2013 11:05, Ferrous Cranus wrote: > Στις 2/9/2013 3:21 μμ, ο/η Dave Angel έγραψε: >> Starting with the byte string in the error message: > f = open("junk.txt", "w") > f.write(b'\xb6\xe3\xed\xf9\xf3\xf4\xef\xfc\xed\xef\xec\xe1 > \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2\n') >

Re: UnicodeDecodeError issue

2013-09-02 Thread Ferrous Cranus
Στις 2/9/2013 3:21 μμ, ο/η Dave Angel έγραψε: Starting with the byte string in the error message: f = open("junk.txt", "w") f.write(b'\xb6\xe3\xed\xf9\xf3\xf4\xef\xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2\n') f.close() Ιndeed but yet again, file checks out the encoding of

Re: UnicodeDecodeError issue

2013-09-02 Thread MRAB
On 02/09/2013 13:24, Dave Angel wrote: On 2/9/2013 07:56, MRAB wrote: On 02/09/2013 12:38, Dave Angel wrote: ¶γνωστοόνομα συστήματος I don't have a clue what it might be; it's not English, and I don't know whatever language it may be in. You don't recognise Greek? I recognize mos

Re: UnicodeDecodeError issue

2013-09-02 Thread Dave Angel
On 2/9/2013 07:56, MRAB wrote: > On 02/09/2013 12:38, Dave Angel wrote: >> ¶γνωστοόνομα συστήματος >> >> I don't have a clue what it might be; it's not English, and I don't >> know whatever language it may be in. >> > You don't recognise Greek? I recognize most of those as Greek characters

Re: UnicodeDecodeError issue

2013-09-02 Thread Dave Angel
On 2/9/2013 07:49, Ferrous Cranus wrote: > Στις 2/9/2013 2:38 μμ, ο/η Dave Angel έγραψε: >> >> Does that string make any sense to you? > > Yes it does, it mean "Unknown Hostname" > >> The Linux 'file' utility thinks this string is in ISO-8859, so you might >> want to try a decode('ISO-8859-1')

Re: UnicodeDecodeError issue

2013-09-02 Thread MRAB
On 02/09/2013 12:38, Dave Angel wrote: On 2/9/2013 00:16, Ferrous Cranus wrote: Have you tried to decode those bytes in various encodings other than utf-8 ? No, because i wasn't aware of what string/variable they were pertaining at. http://pypi.python.org/pypi/chardet is a package which

Re: UnicodeDecodeError issue

2013-09-02 Thread Ferrous Cranus
Στις 2/9/2013 2:38 μμ, ο/η Dave Angel έγραψε: On 2/9/2013 00:16, Ferrous Cranus wrote: Have you tried to decode those bytes in various encodings other than utf-8 ? No, because i wasn't aware of what string/variable they were pertaining at. http://pypi.python.org/pypi/chardet is a p

Re: UnicodeDecodeError issue

2013-09-02 Thread Dave Angel
On 2/9/2013 00:16, Ferrous Cranus wrote: >> >> Have you tried to decode those bytes in various encodings other than >> utf-8 ? > > > No, because i wasn't aware of what string/variable they were pertaining at. > > http://pypi.python.org/pypi/chardet is a package which tries to 'guess' an encod

Re: UnicodeDecodeError issue

2013-09-01 Thread Ferrous Cranus
Στις 2/9/2013 2:14 πμ, ο/η Dave Angel έγραψε: On 1/9/2013 18:23, Ferrous Cranus wrote: i still wonder how come the invalid byte messge dissapeared Too bad you never bothered to narrow it down to its source. if only i knew how up until yesterday when they were appearing. It coul

Re: UnicodeDecodeError issue

2013-09-01 Thread Dave Angel
On 1/9/2013 18:23, Ferrous Cranus wrote: >> > i still wonder how come the invalid byte messge dissapeared > Too bad you never bothered to narrow it down to its source. It could be anywhere on those three lines. If I had to guess, I'd figure it was one of those environment variables. The L

Re: UnicodeDecodeError issue

2013-09-01 Thread Ferrous Cranus
Στις 1/9/2013 7:10 μμ, ο/η Ferrous Cranus έγραψε: Στις 1/9/2013 6:36 μμ, ο/η Dave Angel έγραψε: On 1/9/2013 10:08, Ferrous Cranus wrote: Here is it: errout = open( '/tmp/err.out', 'w' )# opens and truncates the error output file try: gi = pygeoip.GeoIP('/usr/local/share/GeoI

Re: UnicodeDecodeError issue

2013-09-01 Thread Ferrous Cranus
Στις 1/9/2013 6:36 μμ, ο/η Dave Angel έγραψε: On 1/9/2013 10:08, Ferrous Cranus wrote: Here is it: errout = open( '/tmp/err.out', 'w' )# opens and truncates the error output file try: gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat') city = gi.time_zone_by_

Re: UnicodeDecodeError issue

2013-09-01 Thread Dave Angel
On 1/9/2013 09:59, Ferrous Cranus wrote: > Στις 1/9/2013 1:35 μμ, ο/η Dave Angel έγραψε: >> This is my first crack at it (untested): >> >> errout = open("/tmp/err.out", "w") #opens and truncates the error >> output file >> try: >> gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat') >>

Re: UnicodeDecodeError issue

2013-09-01 Thread Dave Angel
On 1/9/2013 10:08, Ferrous Cranus wrote: > Here is it: > > > errout = open( '/tmp/err.out', 'w' ) # opens and truncates the error > output file > try: > gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat') > city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or > g

Re: UnicodeDecodeError issue

2013-09-01 Thread Ferrous Cranus
Στις 1/9/2013 5:08 μμ, ο/η Ferrous Cranus έγραψε: Στις 1/9/2013 11:35 πμ, ο/η Steven D'Aprano έγραψε: On Sat, 31 Aug 2013 23:50:23 -0700, Ferrous Cranus wrote: Τη Σάββατο, 31 Αυγούστου 2013 9:41:27 π.μ. UTC+3, ο χρήστης Ferrous Cranus έγραψε: Suddenly my webiste superhost.gr running my main p

Re: UnicodeDecodeError issue

2013-09-01 Thread Ferrous Cranus
Στις 1/9/2013 11:35 πμ, ο/η Steven D'Aprano έγραψε: On Sat, 31 Aug 2013 23:50:23 -0700, Ferrous Cranus wrote: Τη Σάββατο, 31 Αυγούστου 2013 9:41:27 π.μ. UTC+3, ο χρήστης Ferrous Cranus έγραψε: Suddenly my webiste superhost.gr running my main python script presents me with this error: Code:

Re: UnicodeDecodeError issue

2013-09-01 Thread Ferrous Cranus
Στις 1/9/2013 1:35 μμ, ο/η Dave Angel έγραψε: This is my first crack at it (untested): errout = open("/tmp/err.out", "w") #opens and truncates the error output file try: gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat') city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or g

Re: UnicodeDecodeError issue

2013-09-01 Thread Chris Angelico
On Sun, Sep 1, 2013 at 8:35 PM, Dave Angel wrote: > Your file mode is 'w' which will wipe out anything written earlier. Yet > if the exception does not happen, the file will still contain error(s) > from some earlier run. You should open (and truncate) the file before > the first use, then use t

Re: UnicodeDecodeError issue

2013-09-01 Thread Dave Angel
On 1/9/2013 03:23, Ferrous Cranus wrote: > Στις 1/9/2013 10:12 πμ, ο/η Chris Angelico έγραψε: >> On Sun, Sep 1, 2013 at 4:50 PM, Ferrous Cranus wrote: >>> Ye i'm aware that i need to define variables before i try to make use of >>> them. >>> I have study all of your examples and then re-view my

Re: UnicodeDecodeError issue

2013-09-01 Thread Steven D'Aprano
On Sat, 31 Aug 2013 23:50:23 -0700, Ferrous Cranus wrote: > Τη Σάββατο, 31 Αυγούστου 2013 9:41:27 π.μ. UTC+3, ο χρήστης Ferrous > Cranus έγραψε: >> Suddenly my webiste superhost.gr running my main python script presents >> >> me with this error: >> >> >> >> Code: >> >> UnicodeDecodeError('utf

Re: UnicodeDecodeError issue

2013-09-01 Thread Chris Angelico
On Sun, Sep 1, 2013 at 5:23 PM, Ferrous Cranus wrote: > Let alone that i when i try to set the 'host' variable i get this line at my > '/tmp/err.out' > > > ni...@superhost.gr [~]# cat /tmp/err.out > > UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef > \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf

Re: UnicodeDecodeError issue

2013-09-01 Thread Ferrous Cranus
Στις 1/9/2013 10:12 πμ, ο/η Chris Angelico έγραψε: On Sun, Sep 1, 2013 at 4:50 PM, Ferrous Cranus wrote: Ye i'm aware that i need to define variables before i try to make use of them. I have study all of your examples and then re-view my code and i can *assure* you that the line statement that

Re: UnicodeDecodeError issue

2013-09-01 Thread Chris Angelico
On Sun, Sep 1, 2013 at 4:50 PM, Ferrous Cranus wrote: > Ye i'm aware that i need to define variables before i try to make use of them. > I have study all of your examples and then re-view my code and i can *assure* > you that the line statement that tied to set the 'host' variable is very > earl

Re: UnicodeDecodeError issue

2013-08-31 Thread Ferrous Cranus
Τη Σάββατο, 31 Αυγούστου 2013 9:41:27 π.μ. UTC+3, ο χρήστης Ferrous Cranus έγραψε: > Suddenly my webiste superhost.gr running my main python script presents > > me with this error: > > > > Code: > > UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef > > \xfc\xed\xef\xec\xe1 \xf3\xf

Re: UnicodeDecodeError issue

2013-08-31 Thread Steven D'Aprano
On Sat, 31 Aug 2013 15:58:11 +0300, Ferrous Cranus wrote: Failure is here, line 135: >> cur.execute('''INSERT INTO files (url, host, city, lastvisit) VALUES >> (%s, %s, %s, %s)''', (filename, host, city, lastvisit) ), [...] > But how is this possible since: > > > host = socket.gethostbyaddr( o

Re: UnicodeDecodeError issue

2013-08-31 Thread Ferrous Cranus
Στις 31/8/2013 3:58 μμ, ο/η Ferrous Cranus έγραψε: Στις 31/8/2013 2:28 μμ, ο/η Steven D'Aprano έγραψε: On Sat, 31 Aug 2013 11:31:13 +0300, Ferrous Cranus wrote: Here is the code inside files.py: The code you show is not the ENTIRE code inside of files.py, is it? You are only showing us a sma

Re: UnicodeDecodeError issue

2013-08-31 Thread Ferrous Cranus
Στις 31/8/2013 2:28 μμ, ο/η Steven D'Aprano έγραψε: On Sat, 31 Aug 2013 11:31:13 +0300, Ferrous Cranus wrote: Here is the code inside files.py: The code you show is not the ENTIRE code inside of files.py, is it? You are only showing us a small piece, correct? The code you show: try:

Re: UnicodeDecodeError issue

2013-08-31 Thread Steven D'Aprano
On Sat, 31 Aug 2013 11:31:13 +0300, Ferrous Cranus wrote: > Here is the code inside files.py: The code you show is not the ENTIRE code inside of files.py, is it? You are only showing us a small piece, correct? The code you show: > try: > gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat

Re: UnicodeDecodeError issue

2013-08-31 Thread Ferrous Cranus
Here is the code inside files.py: try: gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat') city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] ) host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] ) or socket.getho

Re: UnicodeDecodeError issue

2013-08-31 Thread Ferrous Cranus
Στις 31/8/2013 10:25 πμ, ο/η Peter Otten έγραψε: Ferrous Cranus wrote: Suddenly my webiste superhost.gr running my main python script presents me with this error: Code: UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2',

Re: UnicodeDecodeError issue

2013-08-31 Thread Peter Otten
Ferrous Cranus wrote: > Suddenly my webiste superhost.gr running my main python script presents > me with this error: > Code: > UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef > \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1, > 'invalid start byte') > Does anyo

Re: UnicodeDecodeError issue

2013-08-31 Thread Ferrous Cranus
Στις 31/8/2013 10:02 πμ, ο/η Ferrous Cranus έγραψε: Στις 31/8/2013 9:53 πμ, ο/η Chris Angelico έγραψε: On Sat, Aug 31, 2013 at 4:41 PM, Ferrous Cranus wrote: Suddenly my webiste superhost.gr running my main python script presents me with this error: Code: UnicodeDecodeError('utf-8', b'\xb6\xe

Re: UnicodeDecodeError issue

2013-08-31 Thread Ferrous Cranus
Στις 31/8/2013 9:53 πμ, ο/η Chris Angelico έγραψε: On Sat, Aug 31, 2013 at 4:41 PM, Ferrous Cranus wrote: Suddenly my webiste superhost.gr running my main python script presents me with this error: Code: UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef \xfc\xed\xef\xec\xe1 \xf3\xf5\x

Re: UnicodeDecodeError issue

2013-08-30 Thread Chris Angelico
On Sat, Aug 31, 2013 at 4:41 PM, Ferrous Cranus wrote: > Suddenly my webiste superhost.gr running my main python script presents me > with this error: > > Code: > UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef > \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1, > 'i

UnicodeDecodeError issue

2013-08-30 Thread Ferrous Cranus
Suddenly my webiste superhost.gr running my main python script presents me with this error: Code: UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1, 'invalid start byte') Does anyone know what this means? -- Web