pytest-2.4.2: colorama on win32, plugin/tmpdir/other fixes
pytest-2.4.2 is another bug-fixing release of the 2.4 series bringing another few fixes and little improvements: - on Windows require colorama and a newer py lib so that py.io.TerminalWriter() now uses colorama instead of its own ctypes hacks. (fixes issue365) thanks Paul Moore for bringing it up. - fix -k matching of tests where repr and attr and other names would cause wrong matches because of an internal implementation quirk (don't ask) which is now properly implemented. fixes issue345. - avoid tmpdir fixture to create too long filenames especially when parametrization is used (issue354) - fix pytest-pep8 and pytest-flakes / pytest interactions (collection names in mark plugin was assuming an item always has a function which is not true for those plugins etc.) Thanks Andi Zeidler. - introduce node.get_marker/node.add_marker API for plugins like pytest-pep8 and pytest-flakes to avoid the messy details of the node.keywords pseudo-dicts. Adapated docs. - remove attempt to dup stdout at startup as it's icky. the normal capturing should catch enough possibilities of tests messing up standard FDs. - add pluginmanager.do_configure(config) as a link to config.do_configure() for plugin-compatibility as usual, docs at http://pytest.org and upgrades via:: pip install -U pytest have fun, holger krekel -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
ANN: psutil 1.1.1 released
=== About === psutil is a module providing an interface for retrieving information on all running processes and system utilization (CPU, memory, disks, network, users) in a portable way by using Python, implementing many functionalities offered by command line tools such as ps, top, free, netstat, lsof and others. It supports Linux, Windows, OSX, FreeBSD and Solaris with Python versions from 2.4 to 3.4. === Bug fixes === This is a bugfix only release fixing high-priority issue #438: https://code.google.com/p/psutil/issues/detail?id=438 This exclusively affected Linux users running kernels 2.6.36 (which includes latest CentOS and Scientific Linux distros). You're encouraged to download new psutil version at: https://pypi.python.org/pypi/psutil#downloads === Links === * Home page: http://code.google.com/p/psutil * Downloads: https://pypi.python.org/pypi?:action=displayname=psutil#downloads * API Reference: http://code.google.com/p/psutil/wiki/Documentation Please try out this new release and let me know if you experience any problem by filing issues on the bug tracker. All the best, --- Giampaolo Rodola' http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Re: Formal-ity and the Church-Turing thesis
On Tuesday, October 8, 2013 10:49:11 AM UTC+5:30, zipher wrote: I don't have an infinite stack to implement lambda calculus, but... And then But this is not a useful formalism. Any particular Program implements a DFA, even as it runs on a TM. The issue of whether than TM is finite or not can be dismissed because a simple calculation can usually suffice, or at least establish a range usefulness so as not to run out of memory. Having it both ways aren't you? -- https://mail.python.org/mailman/listinfo/python-list
Code golf challenge: XKCD 936 passwords
Who's up for some fun? Implement an XKCD-936-compliant password generator in Python 3, in less code than this: print(*__import__(random).sample(open(/usr/share/dict/words).read().split(\n),4)) Second challenge: Use it for generating all your passwords :) [1] https://en.wikipedia.org/wiki/Code_golf [2] http://xkcd.com/936/ ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
On 08/10/2013 06:44, rusi wrote: On Tuesday, October 8, 2013 10:46:50 AM UTC+5:30, Ravi Sahni wrote: With due respect Sir, you saying that Turing machine not a machine? Very confusion Sir!!! Thanks Ravi for the 'due respect' though it is a bit out of place on a list like this :-) With due respect Sir I'd like to point out that this appears to have very little to do (directly) with Python, so to go completely off topic I'll point out that my nephew is currently working on the film about the life of said Alan Turing :) -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On Monday, October 7, 2013 8:17:21 PM UTC-10, Chris Angelico wrote: Who's up for some fun? Implement an XKCD-936-compliant password generator in Python 3, in less code than this: print(*__import__(random).sample(open(/usr/share/dict/words).read().split(\n),4)) Second challenge: Use it for generating all your passwords :) [1] https://en.wikipedia.org/wiki/Code_golf [2] http://xkcd.com/936/ ChrisA Well, here's a start: import random as r print(*r.sample(open(/usr/share/dict/words).readlines(),4)) Shaves off 6 characters. -- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On Monday, October 7, 2013 8:45:39 PM UTC-10, spruce...@gmail.com wrote: On Monday, October 7, 2013 8:17:21 PM UTC-10, Chris Angelico wrote: Who's up for some fun? Implement an XKCD-936-compliant password generator in Python 3, in less code than this: print(*__import__(random).sample(open(/usr/share/dict/words).read().split(\n),4)) Second challenge: Use it for generating all your passwords :) [1] https://en.wikipedia.org/wiki/Code_golf [2] http://xkcd.com/936/ ChrisA Well, here's a start: import random as r print(*r.sample(open(/usr/share/dict/words).readlines(),4)) Shaves off 6 characters. And if we were actually trying then that filename should just be /w. Would get rid of another 19 chars. -- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On Tue, Oct 8, 2013 at 5:48 PM, sprucebond...@gmail.com wrote: And if we were actually trying then that filename should just be /w. Would get rid of another 19 chars. I'm working this on the assumption that the dictionary file already exists (that's where it is on my Debian Linux systems, for instance) and shouldn't be moved :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Multiprocessing and Multithreading
Hi all, Please give me an idea about Multiprocessing and Multithreading. Thanks Regards, Chandru CAUTION - Disclaimer * This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Infosys has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. You should carry out your own virus checks before opening the e-mail or attachment. Infosys reserves the right to monitor and review the content of all messages sent to or from this e-mail address. Messages sent to or from this e-mail address may be stored on the Infosys e-mail system. ***INFOSYS End of Disclaimer INFOSYS*** -- https://mail.python.org/mailman/listinfo/python-list
Re for Apache log file format
Hi,
I am trying to split up the re pattern for Apache log file format and seem
to be having some trouble in getting Python to understand multi-line
pattern:
#!/usr/bin/python
import re
#this is a single line
string = '192.168.122.3 - - [29/Sep/2013:03:52:33 -0700] GET / HTTP/1.0
302 276 - check_http/v1.4.16 (nagios-plugins 1.4.16)'
#trying to break up the pattern match for easy to read code
pattern = re.compile(r'(?Pip\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+'
r'(?Pident\-)\s+'
r'(?Pusername\-)\s+'
r'(?PTZ\[(.*?)\])\s+'
r'(?Purl\(.*?)\)\s+'
r'(?Phttpcode\d{3})\s+'
r'(?Psize\d+)\s+'
r'(?Preferrer\\)\s+'
r'(?Pagent\((.*?)\))')
match = re.search(pattern, string)
if match:
print match.group('ip')
else:
print 'not found'
The python interpreter is skipping to the 'math = re.search' and then the
'if' statement right after it looks at the ip, instead of moving onto
ident and so on.
mybox:~ user$ python -m pdb /Users/user/Documents/Python/apache.py
/Users/user/Documents/Python/apache.py(3)module()
- import re
(Pdb) n
/Users/user/Documents/Python/apache.py(5)module()
- string = '192.168.122.3 - - [29/Sep/2013:03:52:33 -0700] GET /
HTTP/1.0 302 276 - check_http/v1.4.16 (nagios-plugins 1.4.16)'
(Pdb) n
/Users/user/Documents/Python/apache.py(7)module()
- pattern = re.compile(r'(?Pip\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+'
(Pdb) n
/Users/user/Documents/Python/apache.py(17)module()
- match = re.search(pattern, string)
(Pdb)
Thank you.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
Chris Angelico ros...@gmail.com wrote: On Tue, Oct 8, 2013 at 5:48 PM, sprucebond...@gmail.com wrote: And if we were actually trying then that filename should just be /w. Would get rid of another 19 chars. I'm working this on the assumption that the dictionary file already exists (that's where it is on my Debian Linux systems, for instance) and shouldn't be moved :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list Typical MUD Master - making up rules as you go along :-) Sent from a Galaxy far far away-- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On Tue, Oct 8, 2013 at 6:02 PM, Steve Simmons square.st...@gmail.com wrote: Typical MUD Master - making up rules as you go along :-) Totally. Under the auspices of Rule Zero: http://tvtropes.org/pmwiki/pmwiki.php/Main/RuleOfFun :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
random...@fastmail.us writes: The entire point of tail call optimization requires not keeping the intervening stack frames around, in _any_ form, so as to allow arbitrarily deep recursion without ever having the possibility of a stack overflow. An implementation which reduced but did not eliminate the space used per call would not be worthwhile because it would not enable the recursive functional programming patterns that mandatory tail call optimization allows. You could require that an optimizable tail call be made explicit. Actually, I suspect it might be possible to do this now, by abusing exception handling as a control flow mechanism. Python code already marks many of the functionally relevant tail calls with 'return'. It just wants to retain the trace. Another keyword could be used to indicate that the programmer does not want a stack frame retained. It's probably too much to suggest 'please return', but how about 'goto return'? A tail call is a 'goto that passes arguments', and I think 'goto' is a keyword already. (Actually I just wanted to suggest 'please return'. Not seriously.) -- https://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing and Multithreading
On 10/8/2013 1:34 AM, Chandru Rajendran wrote: Please give me an idea about Multiprocessing and Multithreading. Please give us some idea of what you know and what you actually want to know. CAUTION - Disclaimer * This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Infosys has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. You should carry out your own virus checks before opening the e-mail or attachment. Infosys reserves the right to monitor and review the content of all messages sent to or from this e-mail address. Messages sent to or from this e-mail address may be stored on the Infosys e-mail system. ***INFOSYS End of Disclaimer INFOSYS*** And please try to avoid this bulky disclaimer that is a lie. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing and Multithreading
On 08/10/2013 06:34, Chandru Rajendran wrote: Hi all, Please give me an idea about Multiprocessing and Multithreading. Thanks Regards, Chandru I'll assume that you're a newbie so I'll keep it simple. Multiprocessing is about more than one process and multithreading is about more than one thread. If you want Python specifics you could start here http://docs.python.org/3/library/multiprocessing.html or here http://docs.python.org/3/library/threading.html, both of which may have been found by using your favourite search engine. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
Steven D'Aprano writes: Far more useful would be a high-level description of Scheme's programming model. If names can be rebound on the fly, how does Scheme even tell whether something is a recursive call or not? def foo(arg): do stuff here foo(arg-1) # how does Scheme know that this is the same foo? In general, it doesn't know. It just calls whatever function is bound to foo. It does know that the call is in a tail position. If the compiler has access to all code that can possibly change the value of foo, it can know simply by proving that there is no such assignment statement in the code. This can happen if the compiler is told to assume that it has the whole program. It often happens in a local scope. Module systems create such local scopes for unexported variables, and even for exported variables by forbidding assignments outside. (I'm not sure if your question was rhetorical or if you were looking for this information.) -- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On 08/10/2013 07:48, sprucebond...@gmail.com wrote: On Monday, October 7, 2013 8:45:39 PM UTC-10, spruce...@gmail.com wrote: On Monday, October 7, 2013 8:17:21 PM UTC-10, Chris Angelico wrote: Who's up for some fun? Implement an XKCD-936-compliant password generator in Python 3, in less code than this: print(*__import__(random).sample(open(/usr/share/dict/words).read().split(\n),4)) Second challenge: Use it for generating all your passwords :) [1] https://en.wikipedia.org/wiki/Code_golf [2] http://xkcd.com/936/ ChrisA Well, here's a start: import random as r print(*r.sample(open(/usr/share/dict/words).readlines(),4)) Shaves off 6 characters. And if we were actually trying then that filename should just be /w. Would get rid of another 19 chars. Very impressive, you've saved a total of 25 characters on one line and added too many lines to count to your emails, which I've snipped. Please read and digest this https://wiki.python.org/moin/GoogleGroupsPython, thanks in anticipation. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
On Tue, 08 Oct 2013 10:46:50 +0530, Ravi Sahni wrote: On Tue, Oct 8, 2013 at 8:47 AM, rusi rustompm...@gmail.com wrote: I can only say how ironic it sounds to someone who is familiar with the history of our field: Turing was not a computer scientist (the term did not exist then) but a mathematician. And his major contribution was to create a form of argument so much more rigorous than what erstwhile mathematicians were used to that he was justified in calling that math as a machine. The irony is that today's generation assumes that 'some-machine' implies its something like 'Intel-machine'. To get out of this confusion ask yourself: Is it finite or infinite? If the TM were finite it would be a DFA If the Intel-machine (and like) were infinite they would need to exist in a different universe. With due respect Sir, you saying that Turing machine not a machine? Very confusion Sir!!! The mathematical ideal Turing Machine has an infinitely long tape, equivalent to infinite memory, and may take an unbounded amount of time to complete the computation. Since no *actual* physical machine can be infinitely big, and in practice there are strict limits on how long we are willing to wait for a computation to complete, in the *literal* sense, Turing Machines are not *actual* machines. They are a mathematical abstraction. But in practice, we can wave our hands and ignore this fact, and consider only not-quite-Turing Machines with finite amounts of tape, and note that they are equivalent to physical machines with finite amounts of memory. One could even build such a finite Turing Machine, although of course it would be very slow. Or one can simulate it in software. So in that sense, computers are Turing Machines. Anything a physical computing device can compute, a Turing Machine could too. The converse is not true though: a Turing Machine with infinite tape can compute things where a real physical device would run out of memory, although it might take longer than anyone is willing to wait. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: HEX to ASCII
esmaspäev, 7. oktoober 2013 18:52.21 UTC+3 kirjutas Piet van Oostrum:
markot...@gmail.com writes:
This is the code i came up with:
from teisendaja import *
from operator import *
import binascii
teisendus = teisendus()
kood = input(Kood: )
key = input(Võti: )
chunksize = 2
vastus = [teisendus.teisendus3(16,2,kood[i: (i + chunksize)]) for i in
range(0, len(kood),chunksize)]
vastus = [0*(8-len(x)) + x for x in vastus]
#key = teisendus.teisendus3(10,2,int(key))
getBin = lambda x, n: x = 0 and str(bin(x))[2:].zfill(n) or - +
str(bin(x))[3:].zfill(n)
Instead of boolean and expr1 or expr2 in current Python you can better write:
expr1 if boolean else expr2.
and I think using def getBin(x, n):would be more clear.
But I have another observation: You use getBin only for positive ints, so the
whole '-' case isn't necessary. Actually it would be damaging, as the rest of
the code assumes that the key that results from getBin is 8 characters long,
whereas in the negative case it would be 9 characters. Also you use
int(key[j]) and if key[0] == '-' this would give an error.
If you just want to get 8 binary digits for an int using format would be
simpler:
getBin = lambda x, n: '{0:={1}b}'.format(x, n)
or getBin = lambda x, n: '{0:=0{1}b}'.format(x, n) if you want also negatives
(but this would give you 7 or eight binary digits, not always 8.)
def dekrüpteeria(vastus, key):
XOR = []
tulemus = []
for i in range(len(vastus)):
for j in range(8):
XOR.append(str(ixor(int(vastus[i][j]), int(key[j]
tulemus.append(.join(XOR))
key = .join(XOR)
XOR = []
return tulemus
You can use list comprehension:
def dekrüpteeria(vastus, key):
tulemus = []
for i in range(len(vastus)):
XOR = [(str(ixor(int(vastus[i][j]), int(key[j] for j in range(8)]
tulemus.append(.join(XOR))
key = .join(XOR)
return tulemus
and then because you only use .join(XOR), not XOR itself:
def dekrüpteeria(vastus, key):
tulemus = []
for i in range(len(vastus)):
XOR = .join([(str(ixor(int(vastus[i][j]), int(key[j] for j in
range(8)])
tulemus.append(XOR))
key = XOR
return tulemus
and then you could rewrite this also to use a list comprehension for tulemus,
but that may make it in a too big one liner.
Also note that you always use int() on the elements of key and vastus, so it
might be simpler to store these as int arrays (lists) instead of strings
tulemus2= []
if key == :
for i in range(256):
võti = getBin(i,8)
tulemus = [teisendus.teisendus3(2,16,i) for i in
dekrüpteeria(vastus, võti)]
tulemus = [0*(2-len(x)) + x for x in tulemus]
Look at the zfill method for the above 2 line
Probably
tulemus = [teisendus.teisendus3(2,16,i).zfill(2) for i in
dekrüpteeria(vastus, võti)]
will do the same
# for j in range(len(tulemus)):
tulemus2.append(binascii.unhexlify(.join(tulemus)))
print(Key- + str(võti) + : + str(tulemus2))
tulemus2 = []
#tulemus = [teisendus.teisendus3(2,16,i) for i in dekrüpteeria(vastus, key)]
#print(:.join(tulemus))
#751a6f1d3d5c3241365321016c05620a7e5e34413246660461412e5a2e412c49254a24
Although this is quite ugly atm. But it serves me well, until i reach the
unhexlify part. The number and lette r string at the wery end is the mesage
im trying to decypher.
The result of unhexlify is a byte string. So tulemus2 is a list of byte
strings, which you can joint with xxx = b''.join(tulemus2), and then you have
another byte string. If you are sure this is ASCII (which means all bytes are
128), the you can convert it to a string with str(xxx, 'ascii') or
xxx.decode('ascii'). If there are bytes 127 then you have to know which
encoding it is to be able to make a string out of it.
Is this some known encryption method? If so why not use a standard solution
for it? If it is a home brew encryption: are you sure it is safe? Most home
brew solutions in encryption are not.
--
Piet van Oostrum p...@vanoostrum.org
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999
Are you familira with the page called hacker.org? THere are tons of challenges.
And this is one of the cypher challenges called Feedback cypher. The end result
should be in the format of The antswer is . Or something like that. The
problem is that the key is unknown. So i try all the 255 keys there are and
then il look for the line that makes sense.
--
https://mail.python.org/mailman/listinfo/python-list
Re: HEX to ASCII
esmaspäev, 7. oktoober 2013 17:16.29 UTC+3 kirjutas Mark Lawrence: On 07/10/2013 14:54, markot...@gmail.com wrote: I forgot to tell. The teisendaja module that i have imported, is a number converter that allow to convert numbers from one base to another. i mostly use it for HEX to BIN and vice versa, but it supports other bases too. That's nice to know, but what has it got to do with the market price of oranges in Timbuktu? Or to put it another way, you're forcing volunteers to go and find your original message as once again you don't quote any context. Please make life easier for everybody, including yourself, by quoting something from the original. Thanks in anticipation. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence teisendaja module doesent matter. Only thing that matters about is that it returns a string with the converted number. -- https://mail.python.org/mailman/listinfo/python-list
Re: class implementation
I cant just subclassing doesent work. It seem the init method of the source class also calls out another class. And the problem is, i can subclass the other class to with the required function but the end result is that it doesent work, since the source class cant accsess the subclass functions. The source code is pykkar. https://courses.cs.ut.ee/all/MTAT.03.100/2012_fall/uploads/opik/_downloads/pykkar.py I want to add it a new ability called left(). I cant manipulate the source class, cause then my comp will be the only one where the program runs. class pykkar_l(Pykkar): def left(self): self._world.execute(left) def _cmd_left(self): headings = (N,E,S,W) cur_tile = self._get_current_tile() cur_heading_index = headings.index(cur_tile.pykkar_heading) new_heading_index = (cur_heading_index - 1) % 4 cur_tile.pykkar_heading = headings[new_heading_index] self._update_pykkar_image(cur_tile) class world_l(World): def left(self): self._world.execute(left) These are my subclasses. For it to work. Class World, must obtain the method from subclass world_l -- https://mail.python.org/mailman/listinfo/python-list
Re: HEX to ASCII
On 08/10/2013 09:13, markot...@gmail.com wrote: esmaspäev, 7. oktoober 2013 17:16.29 UTC+3 kirjutas Mark Lawrence: On 07/10/2013 14:54, markot...@gmail.com wrote: I forgot to tell. The teisendaja module that i have imported, is a number converter that allow to convert numbers from one base to another. i mostly use it for HEX to BIN and vice versa, but it supports other bases too. That's nice to know, but what has it got to do with the market price of oranges in Timbuktu? Or to put it another way, you're forcing volunteers to go and find your original message as once again you don't quote any context. Please make life easier for everybody, including yourself, by quoting something from the original. Thanks in anticipation. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence teisendaja module doesent matter. Only thing that matters about is that it returns a string with the converted number. Actually another thing that matters is finding a technology that doesn't spread superfluous newlines throughout emails. I've snipped them above. Please take note of this https://wiki.python.org/moin/GoogleGroupsPython. Thanks in aniticipation. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Variable arguments (*args, **kwargs): seeking elegance
On Saturday, October 5, 2013 9:04:25 PM UTC-7, John Ladasky wrote: Hi folks, I'm trying to make some of Python class definitions behave like the ones I find in professional packages, such as Matplotlib. A Matplotlib class can often have a very large number of arguments -- some of which may be optional, some of which will assume default values if the user does not override them, etc. I have working code which does this kind of thing. I define required arguments and their default values as a class attribute, in an OrderedDict, so that I can match up defaults, in order, with *args. I'm using set.issuperset() to see if an argument passed in **kwargs conflicts with one which was passed in *args. I use set.isdisjoint() to look for arguments in **kwargs which are not expected by the class definition, raising an error if such arguments are found. Even though my code works, I'm finding it to be a bit clunky. And now, I'm writing a new class which has subclasses, and so actually keeps the extra kwargs instead of raising an error... This is causing me to re-evaluate my original code. It also leads me to ask: is there a CLEAN and BROADLY-APPLICABLE way for handling the *args/**kwargs/default values shuffle that I can study? Or is this sort of thing too idiosyncratic for there to be a general method? Thanks for any pointers! One thought -- often, people turn to subclassing as the only tool in their toolbox. Have you considered that it may be easier/better to work with delegation and composition instead? Double like. Subclassing is awesome when it is used properly ... which usually means used cautiously. Delegation/composition just doesn't result in the some sort of weird gotchas. -- https://mail.python.org/mailman/listinfo/python-list
Re: class implementation
On 08/10/2013 09:20, markot...@gmail.com wrote: To whom and to what are you replying? -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
random...@fastmail.us writes: On Mon, Oct 7, 2013, at 13:15, Alain Ketterlin wrote: That's fine. My point was: you can't at the same time have full dynamicity *and* procedural optimizations (like tail call opt). Everybody should be clear about the trade-off. Let's be clear about what optimizations we are talking about. Tail call optimization, itself, doesn't care _what_ is being called. It can just as easily mean erase its own stack frame and replace it with that of another function as reassign the arguments and jump to the top of this function. Some people have introduced the idea of _further_ optimizations, transforming near tail recursion (i.e. return self()+1) into tail recursion, and _that_ depends on knowing the identity of the function (though arguably that could be accounted for at the cost of including dead code for the path that assumes it may have been changed), but tail call optimization itself does not. You're right, thanks for the clarification. -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
Antoon Pardon antoon.par...@rece.vub.ac.be writes: Op 07-10-13 19:15, Alain Ketterlin schreef: [...] That's fine. My point was: you can't at the same time have full dynamicity *and* procedural optimizations (like tail call opt). Everybody should be clear about the trade-off. Your wrong. Full dynamics is not in contradiction with tail call optimisation. Scheme has already done it for years. You can rebind names to other functions in scheme and scheme still has working tail call optimisatiosn. See http://en.wikipedia.org/wiki/Scheme_%28programming_language%29#Lexical_scope (first sentence, about variable bindings). -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
Op 08-10-13 01:50, Steven D'Aprano schreef: On Mon, 07 Oct 2013 15:47:26 -0700, Mark Janssen wrote: I challenge you to get down to the machine code in scheme and formally describe how it's doing both. For which machine? Or are you assuming that there's only one machine code that runs on all computing devices? Frankly, asking somebody to *formally* describe a machine code implementation strikes me as confused. Normally formal descriptions are given in terms of abstract operations, often high level operations, sometimes *very* high level, and rarely in terms of low-level flip this bit, copy this byte machine code operations. I'm not sure how one would be expected to generate a formal description of a machine code implementation. But even putting that aside, even if somebody wrote such a description, it would be reductionism gone mad. What possible light on the problem would be shined by a long, long list of machine code operations, even if written using assembly mnemonics? Far more useful would be a high-level description of Scheme's programming model. If names can be rebound on the fly, how does Scheme even tell whether something is a recursive call or not? def foo(arg): do stuff here foo(arg-1) # how does Scheme know that this is the same foo? It doesn't and it doesn't need to. tail call optimisation is not limited to recursive functions. All tail calls can be optimised, recurisive call and others. -- Antoon Pardon -- https://mail.python.org/mailman/listinfo/python-list
inserting or updating appropriately
Hello, i'am trying to insert a new record or update an existing one in
case counterID(stands for the page's URL) and cookieID(random number) is
the same:
try:
# if first time for webpage; create new record( primary key is
automatic, hit is defaulted ), if page exists then update record
cur.execute('''INSERT INTO counters (url) VALUES (%s) ON DUPLICATE KEY
UPDATE hits = hits + 1''', page )
# get the primary key value of the new added record
cID = cur.lastrowid
# if first time visitor on this page, create new record, if visitor
exists then update record
cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city,
useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)
ON DUPLICATE KEY UPDATE cookieID = %s, host = %s, city = %s,
useros = %s, browser = %s, ref = %s, hits = hits + 1, lastvisit = %s
WHERE counterID = %s and cookieID =
%s''',
(cID, cookieID, host, city, useros, browser, ref, lastvisit,
cookieID, host, city, useros, browser, ref, lastvisit, cID, cookieID) )
=
Error is: ProgrammingError(ProgrammingError(1064, You have an error in
your SQL syntax; check the manual that corresponds to your MySQL server
version for the right syntax to use near 'WHERE counterID = 1 and
cookieID = '3815'' at line 3),)
i notticed that if i remove the WHERE clause in the last execute it
works but then its not updating properly.
Can this happen in 1-statemnt with the ON DUPLICATE KEY INVOLVED WITHOUT
BREAKING IT IN IN 2-STATEMNTS?
THANKS.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
On Mon, 07 Oct 2013 20:27:13 -0700, Mark Janssen wrote: But even putting that aside, even if somebody wrote such a description, it would be reductionism gone mad. What possible light on the problem would be shined by a long, long list of machine code operations, even if written using assembly mnemonics? Only that you've got a consistent, stable (and therefore, formalizable) translation from your language to the machine. You are mistaken to think that there is a single, one-to-one, mapping between high-level code and machine code. It's not mistaken. I'm afraid it is. Reality trumps your theory. gcc, clang, Microsoft Visual Studio, and all the many, many other C compilers do not generate identical machine code even when targeting the same hardware. This is a fact. It's not even the case that there is One True Way to implement a particular program on a given hardware device and compilers merely are buggy for doing something else. There are often different ways to implement it which are equally good, the only difference being personal preference. Given a stable and formalized language definition, there should only be continued optimization of the lexical and procedural constructs into better machine code. Better than what? Continued optimization? When you say lexical and procedural constructs, do you mean source code? In the case of an interpreted language like Python (which I'll define as a language which includes a layer of indirection between the user and the machine, Irrelevant. In the case of Python, there is a machine. The fact that it is a VM rather than a physical machine is irrelevant. A machine is a machine -- we could be talking about a Lisp Machine, a Forth Machine, a x86 processor, an Motorola 68000, an Atom processor, one of those old Russian mainframes that used three-state trits instead of two-state bits, or even Babbage's Analytical Engine. Besides, most modern CPUs don't execute machine code directly, they run the machine code in a virtual machine implemented in hardware. So the difference between Python and x86 machine code is just a matter of degree. encouraging the nice benefits of interactivity), such optimization isn't really apropos, because it's not the purpose of python to be optimal to the machine as much as optimal to the programmer. In any case, while such optimization can continue over time, they generally create new compiler releases to indicate such changes. The one-to-one mapping is held by the compiler. Such determinism *defines* the machine, otherwise you might as well get rid of the notion of computer *science*. All else is error, akin to cosmic rays or magic. Unless the source code changes, all else remaining equal, the machine code is supposed to be the same, no matter how many times it is compiled. That is akin to saying that there is *only one* way to measure the speed of light (say), standing in exactly the same place, using exactly the same equipment, using precisely the same measurement techniques, and that if we allow alternative methods for measuring the speed of light, physics is no longer a science. [Only if you use the exact source, compiler, switches, etc]] will the output be the same. And even that is not guaranteed. Oh, and what would cause such non-determinism? The compiler-writer, of course. A compiler is software, and is written by a person, who can program it to do anything the writer wants. If the writer wants the compiler to be non-deterministic, it can be. Some viruses use a similar technique to try to avoid virus scanners. They encrypt the payload, which is functionally equivalent to randomizing it (except it can be reversed if you have the key) so as to defeat virus scanners. A more whimsical example: perhaps a mischievous compiler writer included something like this in her compiler: when compiling integer multiplication, INT * INT: if today is Tuesday: emit machine code that does multiplication using repeated addition otherwise: emit machine code that does multiplication using ADD and SHIFT Both implementations of multiplication are perfectly valid. There may be a performance difference, or there may not be. Since no sensible programming language is going to specify the *detailed* machine code implementation of its high-level operations, such a mischievous compiler would still be valid. Take, for example, the single high-level operation: sort(alist) What machine code will be executed? Obviously that will depend on the sort algorithm used. There are *dozens*. Here are just a few: Well, since you didn't specify your programming language, you're then merely stating an English construct. What difference does it make? But if it will make you feel better, I'm specifying Hypertalk. You've probably never heard of it, but regardless, it exists, and it has a sort command, and the high-level language does not specify which of many sort
Re: inserting or updating appropriately
Στις 8/10/2013 12:15 μμ, ο/η Νίκος Αλεξόπουλος έγραψε:
Hello, i'am trying to insert a new record or update an existing one in
case counterID(stands for the page's URL) and cookieID(random number) is
the same:
try:
# if first time for webpage; create new record( primary key is
automatic, hit is defaulted ), if page exists then update record
cur.execute('''INSERT INTO counters (url) VALUES (%s) ON
DUPLICATE KEY UPDATE hits = hits + 1''', page )
# get the primary key value of the new added record
cID = cur.lastrowid
# if first time visitor on this page, create new record, if
visitor exists then update record
cur.execute('''INSERT INTO visitors (counterID, cookieID, host,
city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s,
%s, %s)
ON DUPLICATE KEY UPDATE cookieID = %s, host =
%s, city = %s, useros = %s, browser = %s, ref = %s, hits = hits + 1,
lastvisit = %s
WHERE counterID = %s and cookieID = %s''',
(cID, cookieID, host, city, useros, browser,
ref, lastvisit, cookieID, host, city, useros, browser, ref, lastvisit,
cID, cookieID) )
=
Error is: ProgrammingError(ProgrammingError(1064, You have an error in
your SQL syntax; check the manual that corresponds to your MySQL server
version for the right syntax to use near 'WHERE counterID = 1 and
cookieID = '3815'' at line 3),)
i notticed that if i remove the WHERE clause in the last execute it
works but then its not updating properly.
Can this happen in 1-statemnt with the ON DUPLICATE KEY INVOLVED WITHOUT
BREAKING IT IN IN 2-STATEMNTS?
THANKS.
Actually what i want is this effect in cur.execute statement:
# if first time visitor on this page, create new record, if visitor
exists then update record
cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city,
useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)''',
(cID, cookieID, host, city, useros,
browser, ref, lastvisit)
cur.execute('''UPDATE visitors SET cookieID = %s, host = %s, city =
%s, useros = %s, browser = %s, ref = %s, hits = hits + 1, lastvisit = %s
WHERE counterID = %s and cookieID =
%s''',
(cookieID, host, city, useros, browser, ref, lastvisit, cID,
cookieID) )
--
What is now proved was at first only imagined! WebHost
http://superhost.gr
--
https://mail.python.org/mailman/listinfo/python-list
Re: inserting or updating appropriately
Νίκος Αλεξόπουλος nikos.gr...@gmail.com writes: Error is: ProgrammingError(ProgrammingError(1064, You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE counterID = 1 and cookieID = '3815'' at line 3),) This is an error from the database server. It has nothing to do with Python, as you already know from previous discussions here. Please do not ask questions about usage of the database server here any more. -- \“Holy knit one purl two, Batman!” —Robin | `\ | _o__) | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
Op 07-10-13 23:27, random...@fastmail.us schreef: On Sat, Oct 5, 2013, at 3:39, Antoon Pardon wrote: What does this mean? Does it mean that a naive implementation would arbitrarily mess up stack traces and he wasn't interested in investigating more sophisticated implementations? Does it mean he just didn't like the idea a stack trace wouldn't be a 100% represenatation of the active call history? Does it mean he investigated more sphisticated implementations but found them to have serious short comings that couldn't be remedied easily? The entire point of tail call optimization requires not keeping the intervening stack frames around, in _any_ form, so as to allow arbitrarily deep recursion without ever having the possibility of a stack overflow. An implementation which reduced but did not eliminate the space used per call would not be worthwhile because it would not enable the recursive functional programming patterns that mandatory tail call optimization allows. So? What about an implementation that would keep its stackframes normally until it deteced recursion had occured. From then on it would only keep what it already had plus the four top stackframes (assuming it are all tail calls for the moment). This would allow for a stacktrace of the last four calls and essentially doesn't need any space per call from then on. -- Antoon Pardon -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
Alain Ketterlin writes: Antoon Pardon writes: Op 07-10-13 19:15, Alain Ketterlin schreef: [...] That's fine. My point was: you can't at the same time have full dynamicity *and* procedural optimizations (like tail call opt). Everybody should be clear about the trade-off. Your wrong. Full dynamics is not in contradiction with tail call optimisation. Scheme has already done it for years. You can rebind names to other functions in scheme and scheme still has working tail call optimisatiosn. See http://en.wikipedia.org/wiki/Scheme_%28programming_language%29#Lexical_scope (first sentence, about variable bindings). # ... Scheme is lexically scoped: all possible variable bindings in a # program unit can be analyzed by reading the text of the program unit # without consideration of the contexts in which it may be called ... The actual procedure to be called is still not known at compile time, in general. It can be a parameter. It needn't even be the value of any explicit variable (needn't be bound to a name). def call(f, a): ... return f(a) # tail call ... def wev(...): ... return (fs if c(k) else gs)[k](a) # tail call ... In the Scheme reports, a variable is said to be bound to a location, which is lexically apparent to the language processor; the value is stored in that location, and assignment to the variable means storing a new value in that location. It works like Python or Java; Python just has a different way of talking about how it works - binding names directly to values in a namespace, and rebinding to different values. However, Scheme processors know that the local variables are not accessible from anywhere else but the local code, so there are more opportunities for compile-time analysis. They can optimize many of those locations away, for example. -- https://mail.python.org/mailman/listinfo/python-list
Cookie gets changed when hit comes from a referrer
# initialize cookie and retrieve cookie from clients browser
cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') )
if cookie.get('ID') is not None:
cookieID = cookie['ID'].value
else:
cookieID = random.randrange(0, )
cookie['ID'] = cookieID
cookie['ID']['path'] = '/'
print( cookie )
===
===
I use this code to retrive or set a cookie to the visitor's browser if
present and identify him bu it.
All work well except the situation where the user visits my webpage by
clicking a backlink on another wbpage.
Then for some reason the cookieID changes to another value thus a new
entry appears into the database when insert happens.
What cna i do about that?
--
https://mail.python.org/mailman/listinfo/python-list
class-private names and the Zen of Python
In the following case: class Foo: ... _Foo__a = 100 ... __a = 33 ... Foo._Foo__a 33 I think this behavior, for a user who does not know the convention, could be a surprise. Should be raising an exception (in order to inform the user the transformation of the name __a have been replaced an existing name) a possible --explicit-- alternative? Another question is: where is the place in which this transformation occurs? Is it at the parser level, before the dictionary attribute is gave as argument to the metaclass? I looked at the documentation: http://docs.python.org/3/reference/lexical_analysis.html http://docs.python.org/3/reference/expressions.html#atom-identifiers but it is not clear when this transformation happens. -- Marco Buttu -- https://mail.python.org/mailman/listinfo/python-list
Re: Re for Apache log file format
On 08.10.2013 08:33, Sam Giraffe wrote:
#this is a single line
string = '192.168.122.3 - - [29/Sep/2013:03:52:33 -0700] GET / HTTP/1.0
302 276 - check_http/v1.4.16 (nagios-plugins 1.4.16)'
#trying to break up the pattern match for easy to read code
pattern = re.compile(r'(?Pip\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+'
r'(?Pident\-)\s+'
r'(?Pusername\-)\s+'
r'(?PTZ\[(.*?)\])\s+'
r'(?Purl\(.*?)\)\s+'
r'(?Phttpcode\d{3})\s+'
r'(?Psize\d+)\s+'
r'(?Preferrer\\)\s+'
r'(?Pagent\((.*?)\))')
[SNIP]
The python interpreter is skipping to the 'math = re.search' and then the
'if' statement right after it looks at the ip, instead of moving onto
ident and so on.
I'm not sure if I understand your problem, but your regex pattern only
matches up to the size. When you look for the referrer, the pattern
expects two quotes but in your string you have - (quote, dash, quote).
Thus there is no match (i.e. match is None) and the if-statement will
print not found.
Bye, Andreas
--
https://mail.python.org/mailman/listinfo/python-list
Re: class implementation
On 8/10/2013 04:20, markot...@gmail.com wrote: I cant just subclassing doesent work. I can't parse that sentence. It seem the init method of the source class also calls out another class. And the problem is, i can subclass the other class to with the required function but the end result is that it doesent work, since the source class cant accsess the subclass functions. What's a source class? If you mean parent class, then say so. Otherwise, if you give it a name, we might be able to follow. But source and other don't narrow the field very much. A parent class can certainly access the child class (subclass) methods (not functions). But only if the instance (self) is an instance of the child class. That's the whole point of subclassing. The source code is pykkar. https://courses.cs.ut.ee/all/MTAT.03.100/2012_fall/uploads/opik/_downloads/pykkar.py I want to add it a new ability called left(). I cant manipulate the source class, cause then my comp will be the only one where the program runs. class pykkar_l(Pykkar): def left(self): self._world.execute(left) def _cmd_left(self): headings = (N,E,S,W) cur_tile = self._get_current_tile() cur_heading_index = headings.index(cur_tile.pykkar_heading) new_heading_index = (cur_heading_index - 1) % 4 cur_tile.pykkar_heading = headings[new_heading_index] self._update_pykkar_image(cur_tile) class world_l(World): def left(self): self._world.execute(left) These are my subclasses. For it to work. Class World, must obtain the method from subclass world_l Then it sounds like you should make sure that the global value world in that module is an instance of your world_l class, rather than an instance or World. And that the proxy is an instance of pykkar_l rather than of Pykkar. import pykkar layout = fdlkjdsljdslfkjsdljfdsf pykkar.world = world_I(layout) ??? = pykkar_l(pykkar.world) You don't show your own top-level code, so I can't integrate it in. By the way, it's conventional to use uppercase for class names, and lowercase for instances of those classes. I'm astounded that your class is using eval and multiprocessing before understanding classes and subclasses. -- DaveA -- https://mail.python.org/mailman/listinfo/python-list
Re: class-private names and the Zen of Python
On 10/8/2013 6:13 AM, Marco Buttu wrote: In the following case: class Foo: ... _Foo__a = 100 ... __a = 33 ... Foo._Foo__a 33 I think this behavior, for a user who does not know the convention, could be a surprise. No one qualified to use such names would do such a thing , so there is no need to worry about it or do anything about it. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: class-private names and the Zen of Python
On 10/08/2013 12:36 PM, Terry Reedy wrote: On 10/8/2013 6:13 AM, Marco Buttu wrote: In the following case: class Foo: ... _Foo__a = 100 ... __a = 33 ... Foo._Foo__a 33 I think this behavior, for a user who does not know the convention, could be a surprise. No one qualified to use such names would do such a thing , so there is no need to worry about it or do anything about it. Is this transformation performed by the parser, before to call the metaclass? -- Marco Buttu -- https://mail.python.org/mailman/listinfo/python-list
Re: class-private names and the Zen of Python
On 10/8/13 6:13 AM, Marco Buttu wrote: In the following case: class Foo: ... _Foo__a = 100 ... __a = 33 ... Foo._Foo__a 33 I think this behavior, for a user who does not know the convention, could be a surprise. Should be raising an exception (in order to inform the user the transformation of the name __a have been replaced an existing name) a possible --explicit-- alternative? You also get a problem if you do this: class Foo: ... a = 100 ... a = 33 ... Foo.a 33 Or for that matter: a = 100 a = 33 a 33 There are lots of ways to change what value a name refers to, it's not an error to reassign names. Also, as Terry mentions, no one has ever assigned the two names you show, so why try to warn about it? --Ned. -- https://mail.python.org/mailman/listinfo/python-list
Re: Cookie gets changed when hit comes from a referrer
On Tue, Oct 8, 2013 at 4:04 AM, Νίκος Αλεξόπουλος nikos.gr...@gmail.com wrote: I use this code to retrive or set a cookie to the visitor's browser if present and identify him bu it. All work well except the situation where the user visits my webpage by clicking a backlink on another wbpage. Then for some reason the cookieID changes to another value thus a new entry appears into the database when insert happens. What cna i do about that? This question is really about HTTP, not Python, so you'd have better luck asking elsewhere. The most likely possibility is that the domain doesn't match. For example, the cookie is set for the domain www.foo.com, and the other webpage is linking to foo.com. Another possibility is that the cookie is expiring because the browser session was terminated, not because of anything to do with the other webpage. Or it could simply be a bug or unusual setting in whatever browser you're using to test it. -- https://mail.python.org/mailman/listinfo/python-list
Re: class-private names and the Zen of Python
On 10/08/2013 01:07 PM, Ned Batchelder wrote: On 10/8/13 6:13 AM, Marco Buttu wrote: class Foo: ... _Foo__a = 100 ... __a = 33 ... Foo._Foo__a 33 ... You also get a problem if you do this: class Foo: ... a = 100 ... a = 33 ... Foo.a 33 But this does not happen under the hood, it is explicit Also, as Terry mentions, no one has ever assigned the two names you show, Sincerely, I can not now if someone has assigned (or will assegne) in such way... -- Marco Buttu -- https://mail.python.org/mailman/listinfo/python-list
parsing email from stdin
I want to do some postprocessing on messages from a particular mailbox. So I use getmail which will fetch the messages and feed them to stdin of my program. As I don't know what encoding these messages will be in, I thought it would be prudent to read stdin as binary data. Using python 3.3 on a debian box I have the following code. #!/usr/bin/python3 import sys from email import message_from_file sys.stdin = sys.stdin.detach() msg = message_from_file(sys.stdin) which gives me the following trace back File /home/apardon/.getmail/verdeler, line 7, in module msg = message_from_file(sys.stdin) File /usr/lib/python3.3/email/__init__.py, line 56, in message_from_file return Parser(*args, **kws).parse(fp) File /usr/lib/python3.3/email/parser.py, line 58, in parse feedparser.feed(data) File /usr/lib/python3.3/email/feedparser.py, line 167, in feed self._input.push(data) File /usr/lib/python3.3/email/feedparser.py, line 100, in push data, self._partial = self._partial + data, '' TypeError: Can't convert 'bytes' object to str implicitly)) which seems to be rather odd. The following header are in the msg: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit So why doesn't the email parser lookup the charset and use that for converting to string type? What is the canonical way to parse an email message from stdin? -- Antoon Pardon -- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
In article mailman.830.1381215134.18130.python-l...@python.org, Chris Angelico ros...@gmail.com wrote: On Tue, Oct 8, 2013 at 5:48 PM, sprucebond...@gmail.com wrote: And if we were actually trying then that filename should just be /w. Would get rid of another 19 chars. I'm working this on the assumption that the dictionary file already exists (that's where it is on my Debian Linux systems, for instance) and shouldn't be moved :) ChrisA In the old days, it used to be /usr/dict/words. Port Python to v6, and save another 6 characters :-) -- https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
On Tue, Oct 8, 2013 at 1:20 PM, Steven D'Aprano st...@pearwood.info wrote: On Tue, 08 Oct 2013 10:46:50 +0530, Ravi Sahni wrote: On Tue, Oct 8, 2013 at 8:47 AM, rusi rustompm...@gmail.com wrote: I can only say how ironic it sounds to someone who is familiar with the history of our field: Turing was not a computer scientist (the term did not exist then) but a mathematician. And his major contribution was to create a form of argument so much more rigorous than what erstwhile mathematicians were used to that he was justified in calling that math as a machine. The irony is that today's generation assumes that 'some-machine' implies its something like 'Intel-machine'. To get out of this confusion ask yourself: Is it finite or infinite? If the TM were finite it would be a DFA If the Intel-machine (and like) were infinite they would need to exist in a different universe. With due respect Sir, you saying that Turing machine not a machine? Very confusion Sir!!! The mathematical ideal Turing Machine has an infinitely long tape, equivalent to infinite memory, and may take an unbounded amount of time to complete the computation. Since no *actual* physical machine can be infinitely big, and in practice there are strict limits on how long we are willing to wait for a computation to complete, in the *literal* sense, Turing Machines are not *actual* machines. They are a mathematical abstraction. But in practice, we can wave our hands and ignore this fact, and consider only not-quite-Turing Machines with finite amounts of tape, and note that they are equivalent to physical machines with finite amounts of memory. One could even build such a finite Turing Machine, although of course it would be very slow. Or one can simulate it in software. So in that sense, computers are Turing Machines. Anything a physical computing device can compute, a Turing Machine could too. The converse is not true though: a Turing Machine with infinite tape can compute things where a real physical device would run out of memory, although it might take longer than anyone is willing to wait. Thanks Sir the detailed explanation. You are offering me many thoughts inside few words so I will need some time to meditate upon the same. Presently Sir, I wish to ask single question: What you mean wave our hands?? Thanks -- Ravi -- https://mail.python.org/mailman/listinfo/python-list
Re: Re for Apache log file format
On 2013-10-08, Sam Giraffe s...@giraffetech.biz wrote:
Hi,
I am trying to split up the re pattern for Apache log file format and seem
to be having some trouble in getting Python to understand multi-line
pattern:
#!/usr/bin/python
import re
#this is a single line
string = '192.168.122.3 - - [29/Sep/2013:03:52:33 -0700] GET / HTTP/1.0
302 276 - check_http/v1.4.16 (nagios-plugins 1.4.16)'
#trying to break up the pattern match for easy to read code
pattern = re.compile(r'(?Pip\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+'
r'(?Pident\-)\s+'
r'(?Pusername\-)\s+'
r'(?PTZ\[(.*?)\])\s+'
r'(?Purl\(.*?)\)\s+'
r'(?Phttpcode\d{3})\s+'
r'(?Psize\d+)\s+'
r'(?Preferrer\\)\s+'
r'(?Pagent\((.*?)\))')
I recommend using the re.VERBOSE flag when explicating an re.
It'll make your life incrementally easier.
pattern = re.compile(
r(?Pip\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+
(?Pident\-)\s+
(?Pusername\-)\s+
(?PTZ\[(.*?)\])\s+# You can even insert comments.
(?Purl\(.*?)\)\s+
(?Phttpcode\d{3})\s+
(?Psize\d+)\s+
(?Preferrer\\)\s+
(?Pagent\((.*?)\)), re.VERBOSE)
--
Neil Cerutti
--
https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
On Tue, Oct 8, 2013 at 11:14 AM, rusi rustompm...@gmail.com wrote: To explain at length will be too long and OT (off-topic) for this list. I'll just give you a link and you tell me what you make of it: http://sloan.stanford.edu/mousesite/Secondary/Whorfframe2.html I am trying to read link. Very new idea: Buildings can catch fire by wrong boards!! Later part difficult for me to read. (My English not powerful --please excuse.) I will make my fullest efforts to read on your recommend but I not clear the connection with computers, programming, computer science and so on. Also this Mr. Mark Lawrence question. -- Ravi -- https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
On Tue, 08 Oct 2013 18:16:01 +0530, Ravi Sahni wrote: So in that sense, computers are Turing Machines. Anything a physical computing device can compute, a Turing Machine could too. The converse is not true though: a Turing Machine with infinite tape can compute things where a real physical device would run out of memory, although it might take longer than anyone is willing to wait. Thanks Sir the detailed explanation. You are offering me many thoughts inside few words so I will need some time to meditate upon the same. Presently Sir, I wish to ask single question: What you mean wave our hands?? It is an idiom very common in Australia. (It may not be well known in the rest of the English-speaking world.) It means to figuratively flap one's hands around in the air while skipping over technical details or complications. For example, we often talk about hand-wavy estimates for how long a job will take: my hand-wavy estimate is it will take two days is little better than a guess. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: class-private names and the Zen of Python
On Tue, 08 Oct 2013 12:13:48 +0200, Marco Buttu wrote:
In the following case:
class Foo:
... _Foo__a = 100
... __a = 33
...
Foo._Foo__a
33
I think this behavior, for a user who does not know the convention,
could be a surprise.
Yes, you are correct. It surprised me, and I've been using Python for
more than 15 years, and I know the convention of double-underscore name-
mangling.
Should be raising an exception (in order to inform
the user the transformation of the name __a have been replaced an
existing name) a possible --explicit-- alternative?
No, I don't think so. That would slow down class creation, for no real
benefit. Except for the name-mangling part, this is no different from:
class Spam:
x = 23
x = 42
If anything, something like PyLint or PyChecker could warn about it. But
the language itself is fine like it is.
Another question is: where is the place in which this transformation
occurs? Is it at the parser level, before the dictionary attribute is
gave as argument to the metaclass?
Good question!
I don't have a full answer, but I have a part answer: it occurs before
the metaclass sees the namespace:
py class Meta(type):
... def __new__(meta, name, bases, namespace):
... print(namespace)
... return super().__new__(meta, name, bases, namespace)
...
py
py class Test(metaclass=Meta):
... __test = 'foo'
...
{'__module__': '__main__', '_Test__test': 'foo', '__qualname__': 'Test'}
so I think it is done by the parser.
--
Steven
--
https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
On Tuesday, October 8, 2013 6:31:21 PM UTC+5:30, Ravi Sahni wrote: On Tue, Oct 8, 2013 at 11:14 AM, rusi wrote: To explain at length will be too long and OT (off-topic) for this list. I'll just give you a link and you tell me what you make of it: http://sloan.stanford.edu/mousesite/Secondary/Whorfframe2.html I am trying to read link. Very new idea: Buildings can catch fire by wrong boards!! Later part difficult for me to read. (My English not powerful --please excuse.) I will make my fullest efforts to read on your recommend Hell No! I only asked you to read the first page! [And 'Mr. Mark' will scoldwink] but I not clear the connection with computers, programming, computer science and so on. Also this Mr. Mark Lawrence question. Once you get that buildings can catch fire by wrong terminology you should get that: - the term 'Turing machine' can make people think its a machine even though its a mathematical formalism - the term 'λ-calculus' (partly due to the word calculus and partly due to the greek lambda) makes people think its mathematics even though its a computational framework -- https://mail.python.org/mailman/listinfo/python-list
Encoding of surrogate code points to UTF-8
I think this is a bug in Python's UTF-8 handling, but I'm not sure.
If I've read the Unicode FAQs correctly, you cannot encode *lone*
surrogate code points into UTF-8:
http://www.unicode.org/faq/utf_bom.html#utf8-5
Sure enough, using Python 3.3:
py surr = '\udc80'
py surr.encode('utf-8')
Traceback (most recent call last):
File stdin, line 1, in module
UnicodeEncodeError: 'utf-8' codec can't encode character '\udc80' in
position 0: surrogates not allowed
But reading the previous entry in the FAQs:
http://www.unicode.org/faq/utf_bom.html#utf8-4
I interpret this as meaning that I should be able to encode valid pairs
of surrogates. So if I find a code point that encodes to a surrogate pair
in UTF-16:
py c = '\N{LINEAR B SYLLABLE B038 E}'
py surr_pair = c.encode('utf-16be')
py print(surr_pair)
b'\xd8\x00\xdc\x01'
and then use those same values as the code points, I ought to be able to
encode to UTF-8, as if it were the same \N{LINEAR B SYLLABLE B038 E} code
point. But I can't:
py s = '\ud800\udc01'
py s.encode('utf-8')
Traceback (most recent call last):
File stdin, line 1, in module
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in
position 0: surrogates not allowed
Have I misunderstood? I think that Python is being too strict about
rejecting surrogate code points. It should only reject lone surrogates,
or invalid pairs, not valid pairs. Have I misunderstood the Unicode FAQs,
or is this a bug in Python's handling of UTF-8?
--
Steven
--
https://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing and Multithreading
On 2013-10-08, Mark Lawrence breamore...@yahoo.co.uk wrote: On 08/10/2013 06:34, Chandru Rajendran wrote: Hi all, Please give me an idea about Multiprocessing and Multithreading. Thanks Regards, Chandru I'll assume that you're a newbie so I'll keep it simple. Multiprocessing is about more than one process and multithreading is about more than one thread. I doubt a newbie knows the difference between a thread and a process. Threads share all memory and global variabls. They can communicate with each other through global variables. Processes are completely isolated from each other and much communicate with each other through mechanisms provided by the OS (e.g. sockets, mailboxes, pipes, files). -- Grant Edwards grant.b.edwardsYow! Are the STEWED PRUNES at still in the HAIR DRYER? gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: class implementation
Parent class is at the link. -- https://mail.python.org/mailman/listinfo/python-list
Re: Image manipulation
First helpful advice i have gotten from this forum -- https://mail.python.org/mailman/listinfo/python-list
Re: Cookie gets changed when hit comes from a referrer
Στις 8/10/2013 2:08 μμ, ο/η Ian Kelly έγραψε: On Tue, Oct 8, 2013 at 4:04 AM, Νίκος Αλεξόπουλος nikos.gr...@gmail.com wrote: I use this code to retrive or set a cookie to the visitor's browser if present and identify him bu it. All work well except the situation where the user visits my webpage by clicking a backlink on another wbpage. Then for some reason the cookieID changes to another value thus a new entry appears into the database when insert happens. What cna i do about that? This question is really about HTTP, not Python, so you'd have better luck asking elsewhere. The most likely possibility is that the domain doesn't match. For example, the cookie is set for the domain www.foo.com, and the other webpage is linking to foo.com. Another possibility is that the cookie is expiring because the browser session was terminated, not because of anything to do with the other webpage. Or it could simply be a bug or unusual setting in whatever browser you're using to test it. When i direct hit http://superhost.gr the cookie remains the same it is not lost. Also i have set: ookie['ID']['expires'] = 60*60*24*365 #this cookie will expire in a year but that didnt also help much because the cookie is also changing when the hit comes through a referrer. So, i cannot se the cookie down to its feet my whole insert or update procedure breaks and i have duplicate entried for the same hostnames. Where shoudl i rely to identify a visitor? I was relying on tis hostname(although i know that after a router reset) it changes, and then couple days ago i was thiking of relying to a cookie that i would/set retrive from the vistirs browser, but if it changes all the time i cannot evan rely to that. -- What is now proved was at first only imagined! WebHost http://superhost.gr -- https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
On 08/10/13 14:11, Steven D'Aprano wrote: On Tue, 08 Oct 2013 18:16:01 +0530, Ravi Sahni wrote: Presently Sir, I wish to ask single question: What you mean wave our hands?? It is an idiom very common in Australia. (It may not be well known in the rest of the English-speaking world.) It means to figuratively flap one's hands around in the air while skipping over technical details or complications. It's known elsewhere as well (though mostly in technical circles) - it's in the Jargon File as http://www.catb.org/jargon/html/H/handwave.html. http://www.catb.org/jargon/html/H/handwave.html http://www.catb.org/jargon/html/H/handwave.html -- https://mail.python.org/mailman/listinfo/python-list
converting letters to numbers
I have to define a function add(c1, c2), where c1 and c2 are capital letters;
the return value should be the sum (obtained by converting the letters to
numbers, adding mod 26, then converting back to a capital letter).
All I have so far is:
def add(c1, c2):
ord(c1) - ord('a') + 1
ord(c2) - ord('a') + 1
I know I need to use ord and chr, just not sure how.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Image manipulation
On Tue, 08 Oct 2013 07:15:46 -0700, markotaht wrote: First helpful advice i have gotten from this forum If you insist on dropping cryptic comments with bad spelling, incoherent sentences, and a complete lack of any context, it might be the last advice you get too. Please help us to help you. We are not mind readers, we cannot read your mind and magically understand what you are talking about. Please include content when replying to an posts. Please take the time to try to explain your questions, with proper grammar and syntax and spelling. We will make allowances if English is not your native language, but we won't make allowances if you are just being lazy. Please show small code snippets that demonstrate the problem. You should read this site: even though it is written for Java, the basic ideas hold for Python as well. http://sscce.org Remember that we are volunteers and we are not being paid to help you. The harder you make it for us to understand your posts, the less likely we are to solve your problem. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: parsing email from stdin
On 08.10.2013 14:20, Antoon Pardon wrote: As I don't know what encoding these messages will be in, I thought it would be prudent to read stdin as binary data. Using python 3.3 on a debian box I have the following code. #!/usr/bin/python3 import sys from email import message_from_file sys.stdin = sys.stdin.detach() msg = message_from_file(sys.stdin) Looking at the docs, I've found there is also message_from_binary_file which works for me with your code. http://docs.python.org/3/library/email.parser.html#email.message_from_binary_file Bye, Andreas -- https://mail.python.org/mailman/listinfo/python-list
Re: Cookie gets changed when hit comes from a referrer
Στις 8/10/2013 2:08 μμ, ο/η Ian Kelly έγραψε: This question is really about HTTP, not Python, so you'd have better luck asking elsewhere. The most likely possibility is that the domain doesn't match. For example, the cookie is set for the domain www.foo.com, and the other webpage is linking to foo.com. I think this is the problem but iam not sure entirely how you mean. Can you please explain it a bit more? Shall i change cookie['ID']['path'] = '/' to something else so that never happens? -- What is now proved was at first only imagined! WebHost http://superhost.gr -- https://mail.python.org/mailman/listinfo/python-list
Re: converting letters to numbers
On 08/10/13 15:28, kjaku...@gmail.com wrote:
I have to define a function add(c1, c2), where c1 and c2 are capital letters;
the return value should be the sum (obtained by converting the letters to
numbers, adding mod 26, then converting back to a capital letter).
Can you give some expected outputs? For example, add('A', 'B') should
presumably return 'C', and add('M', 'B') should presumably return 'O',
but what about add('A', 'A') or add('Z', 'Z')?
It feels like the only tricky bit is mapping letters to numbers (i.e.
does A equal 1 or 0?), which you'd do by subtracting a fixed value from
the result of chr. Once you've done that, you'd do the arithmetic to get
a number between 1 and 26 (or 0 and 25), then add the same fixed value
to that and call ord on the result.
--
https://mail.python.org/mailman/listinfo/python-list
Re: JUST GOT HACKED
I don't think you are allowed to use the word dumbass to describe anyone or anything buddy. On Tuesday, October 1, 2013 9:42:31 AM UTC-4, Ferrous Cranus wrote: Στις 1/10/2013 4:27 μμ, ο/η Chris “Kwpolska” Warrick έγραψε: On Tue, Oct 1, 2013 at 3:15 PM, Νίκος nikos.gr...@gmail.com wrote: Στις 1/10/2013 4:06 μμ, ο/η Mark Lawrence έγραψε: On 01/10/2013 10:58, Νίκος wrote: Just logged in via FTP to my server and i saw an uploade file named Warnign html Contents were: WARNING I am incompetent. Do not hire me! Question: WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY ACCOUNT? PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! Would you please stop posting, I've almost burst my stomach laughing at this. You definetely have a ready made career writing comedy. Okey smartass, Try to do it again, if you be successfull again i'll even congratulate you myself. -- https://mail.python.org/mailman/listinfo/python-list It looks like you are accusing someone of doing something without any proof whatsoever. Would you like help with the fallout of the lawsuit that I hope Mark might (should!) come up with?i'am Speaking of “try again”, I doubt it would be hard… As long as a FTP daemon is running somewhere (and you clearly do not know better); or even you have a SSH daemon and you do not know better, an attacker can: a) wait for you to publish your password yet again; b) get you to download an exploit/keylogger/whatever; c) brute-force. Well, considering it’s unlikely you actually have a long-as-shit password, (c) is the best option. Unless your password is very long, in which case is not. I’m also wondering what language your password is in. If you actually used a Greek phrase, how long will it take you to get locked out due to encoding bullshit? Like i use grek letter for my passwords or like i'am gonna fall for any of your 3 dumbass reasons. I already foudn the weakness and corrected it. -- https://mail.python.org/mailman/listinfo/python-list
Re: converting letters to numbers
On Tuesday, October 8, 2013 10:47:39 AM UTC-4, Robert Day wrote:
On 08/10/13 15:28, kjaku...@gmail.com wrote:
Can you give some expected outputs? For example, add('A', 'B') should
presumably return 'C', and add('M', 'B') should presumably return 'O',
but what about add('A', 'A') or add('Z', 'Z')?
It feels like the only tricky bit is mapping letters to numbers (i.e.
does A equal 1 or 0?), which you'd do by subtracting a fixed value from
the result of chr. Once you've done that, you'd do the arithmetic to get
a number between 1 and 26 (or 0 and 25), then add the same fixed value
to that and call ord on the result.
Expected output is add('C', 'E') returns 'G'; where 'C' and 'E' correspond to 2
and 4 respectively with sum 6, corresponding to 'G'.
--
https://mail.python.org/mailman/listinfo/python-list
Re: JUST GOT HACKED
From what I gather he was viewing files uploaded to the ftp folder and found this warning.html file contained within... So my take on it is, someone just uploaded it and this guy is freaking out making a buffoon out of himself. -- https://mail.python.org/mailman/listinfo/python-list
Re: Image manipulation
teisipäev, 8. oktoober 2013 17:26.33 UTC+3 kirjutas Steven D'Aprano: On Tue, 08 Oct 2013 07:15:46 -0700, markotaht wrote: First helpful advice i have gotten from this forum If you insist on dropping cryptic comments with bad spelling, incoherent sentences, and a complete lack of any context, it might be the last advice you get too. Please help us to help you. We are not mind readers, we cannot read your mind and magically understand what you are talking about. Please include content when replying to an posts. Please take the time to try to explain your questions, with proper grammar and syntax and spelling. We will make allowances if English is not your native language, but we won't make allowances if you are just being lazy. Please show small code snippets that demonstrate the problem. You should read this site: even though it is written for Java, the basic ideas hold for Python as well. http://sscce.org Remember that we are volunteers and we are not being paid to help you. The harder you make it for us to understand your posts, the less likely we are to solve your problem. -- Steven Well english isnt my native language, and there are things i just dont know how to explain in any language. And i cant give all the codes i am using, since there are loads of pieces i do not own, but i know the people who made them. -- https://mail.python.org/mailman/listinfo/python-list
Re: Image manipulation
I rembembered a bit too late, are there any good tutorials for Pillow? ImageTk.PhotoImage() keeps giving me error that there isnt such a method. -- https://mail.python.org/mailman/listinfo/python-list
Re: converting letters to numbers
You wrote this:
def add(c1, c2):
ord(c1) - ord('a') + 1
ord(c2) - ord('a') + 1
First of all, this looks like homework. People will help you with
concepts here, but most frown on just providing answers. With that in
mind look at this:
ord('A')
65
ord('a')
97
In your assignment you refer to Upper case letters. In your code you
take the ordinal value of lower case 'a'
--
Joel Goldstick
http://joelgoldstick.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Encoding of surrogate code points to UTF-8
On 2013-10-08, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote:
py c = '\N{LINEAR B SYLLABLE B038 E}'
py surr_pair = c.encode('utf-16be')
py print(surr_pair)
b'\xd8\x00\xdc\x01'
and then use those same values as the code points, I ought to be able to
encode to UTF-8, as if it were the same \N{LINEAR B SYLLABLE B038 E} code
point. But I can't:
py s = '\ud800\udc01'
py s.encode('utf-8')
Traceback (most recent call last):
File stdin, line 1, in module
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in
position 0: surrogates not allowed
Have I misunderstood? I think that Python is being too strict
about rejecting surrogate code points. It should only reject
lone surrogates, or invalid pairs, not valid pairs. Have I
misunderstood the Unicode FAQs, or is this a bug in Python's
handling of UTF-8?
From RFC 3629:
The definition of UTF-8 prohibits encoding character numbers
between U+D800 and U+DFFF, which are reserved for use with the
UTF-16 encoding form (as surrogate pairs) and do not directly
represent characters. When encoding in UTF-8 from UTF-16 data,
it is necessary to first decode the UTF-16 data to obtain
character numbers, which are then encoded in UTF-8 as described
above. This contrasts with CESU-8 [CESU-8], which is a
UTF-8-like encoding that is not meant for use on the Internet.
CESU-8 operates similarly to UTF-8 but encodes the UTF-16 code
values (16-bit quantities) instead of the character number
(code point). This leads to different results for character
numbers above 0x; the CESU-8 encoding of those characters
is NOT valid UTF-8.
The Wikipedia article points out:
Whether an actual application should [refuse to encode these
character numbers] is debatable, as it makes it impossible to
store invalid UTF-16 (that is, UTF-16 with unpaired surrogate
halves) in a UTF-8 string. This is necessary to store unchecked
UTF-16 such as Windows filenames as UTF-8. It is also
incompatible with CESU encoding (described below).
So Python's interpretation is conformant, though not without some
disadvantages.
In any case, \ud800\udc01 isn't a valid unicode string. In a
perfect world it would automatically get converted to
'\u00010001' without intervention.
--
Neil Cerutti
--
https://mail.python.org/mailman/listinfo/python-list
Re: Encoding of surrogate code points to UTF-8
Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes:
I think this is a bug in Python's UTF-8 handling, but I'm not sure.
[snip]
py s = '\ud800\udc01'
py s.encode('utf-8')
Traceback (most recent call last):
File stdin, line 1, in module
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in
position 0: surrogates not allowed
Have I misunderstood? I think that Python is being too strict about
rejecting surrogate code points. It should only reject lone surrogates,
or invalid pairs, not valid pairs. Have I misunderstood the Unicode FAQs,
or is this a bug in Python's handling of UTF-8?
http://www.unicode.org/versions/Unicode6.2.0/ch03.pdf
D75 Surrogate pair: A representation for a single abstract character
that consists of a sequence of two 16-bit code units, where the first
value of the pair is a high-surrogate code unit and the second value
is a low-surrogate code unit.
* Surrogate pairs are used only in UTF-16. (See Section 3.9, Unicode
EncodingForms.)
* Isolated surrogate code units have no interpretation on their own.
Certain other isolated code units in other encoding forms also have no
interpretation on their own. For example, the isolated byte [\x80] has
no interpretation in UTF-8; it can be used only as part of a multibyte
sequence. (See Table 3-7). It could be argued that this line by itself
should raise an error.
That first bullet indicates that it is indeed illegal to use surrogate
pairs in UTF-8 or UTF-32.
--
Pete Forman
--
https://mail.python.org/mailman/listinfo/python-list
Re: parsing email from stdin
Op 08-10-13 16:24, Andreas Perstinger schreef:
On 08.10.2013 14:20, Antoon Pardon wrote:
As I don't know what encoding these messages will be in, I thought it
would be prudent to read stdin as binary data.
Using python 3.3 on a debian box I have the following code.
#!/usr/bin/python3
import sys
from email import message_from_file
sys.stdin = sys.stdin.detach()
msg = message_from_file(sys.stdin)
Looking at the docs, I've found there is also message_from_binary_file
which works for me with your code.
http://docs.python.org/3/library/email.parser.html#email.message_from_binary_file
I can't try that out right now, but I had a look at the code and the
ByteParser that is mentioned their looks like this:
class BytesFeedParser(FeedParser):
Like FeedParser, but feed accepts bytes.
def feed(self, data):
super().feed(data.decode('ascii', 'surrogateescape'))
Somehow I doubt that trying to decode my utf-8 stream as if it was
ascii will work.
--
Antoon Pardon
--
https://mail.python.org/mailman/listinfo/python-list
Re: converting letters to numbers
On Tue, Oct 8, 2013, at 10:28, kjaku...@gmail.com wrote:
I have to define a function add(c1, c2), where c1 and c2 are capital
letters; the return value should be the sum (obtained by converting the
letters to numbers, adding mod 26, then converting back to a capital
letter).
All I have so far is:
def add(c1, c2):
ord(c1) - ord('a') + 1
ord(c2) - ord('a') + 1
I know I need to use ord and chr, just not sure how.
Your description says capital letters, but 'a' is a lowercase letter.
Does mod 26 means A is 1, or is it 0? i.e., is A+A = B or is it A?
What should your function do if the letter isn't a capital letter from
the basic set of 26 English letters?
--
https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On Tue, 08 Oct 2013 08:33:48 -0400, Roy Smith wrote: In article mailman.830.1381215134.18130.python-l...@python.org, Chris Angelico ros...@gmail.com wrote: On Tue, Oct 8, 2013 at 5:48 PM, sprucebond...@gmail.com wrote: And if we were actually trying then that filename should just be /w. Would get rid of another 19 chars. I'm working this on the assumption that the dictionary file already exists (that's where it is on my Debian Linux systems, for instance) and shouldn't be moved :) In the old days, it used to be /usr/dict/words. Port Python to v6, and save another 6 characters :-) Doesn't matter where it is, a link to it exists at /w now ;) -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: converting letters to numbers
On Tuesday, October 8, 2013 11:36:51 AM UTC-4, rand...@fastmail.us wrote: Your description says capital letters, but 'a' is a lowercase letter. Does mod 26 means A is 1, or is it 0? i.e., is A+A = B or is it A? What should your function do if the letter isn't a capital letter from the basic set of 26 English letters? A is 0. Transfer it to an uppercase letter if it's a letter, if it's not then an error. This isn't right, I know, just testing around def add(c1, c2): ans = '' for i in c1 + c2: ans += chrord(i)-65))%26) + 65) return ans -- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On Tue, Oct 8, 2013, at 2:45, sprucebond...@gmail.com wrote: On Monday, October 7, 2013 8:17:21 PM UTC-10, Chris Angelico wrote: print(*__import__(random).sample(open(/usr/share/dict/words).read().split(\n),4)) # 87 import random as r print(*r.sample(open(/usr/share/dict/words).readlines(),4)) # 80 How about this? My version is also portable to systems with different file locations, and localizable to different language dictionaries (Some assembly required). import sys,random print(*map(str.strip,random.sample(list(sys.stdin),4))) # 73 Importing random as r doesn't actually save anything, since as r is the same five characters you saved from the one use of it. -- https://mail.python.org/mailman/listinfo/python-list
Re: Re for Apache log file format
On Mon, 07 Oct 2013 23:33:31 -0700, Sam Giraffe wrote: I am trying to split up the re pattern for Apache log file format and seem to be having some trouble in getting Python to understand multi-line pattern: Aiui apache log format uses space as delimiter, encapsulates strings in '' characters, and uses '-' as an empty field. So I think every element should match: (\S+|[^]+|-) and there should be \s+ between elements. -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Encoding of surrogate code points to UTF-8
On 2013-10-08, Neil Cerutti ne...@norwich.edu wrote: In any case, \ud800\udc01 isn't a valid unicode string. In a perfect world it would automatically get converted to '\u00010001' without intervention. This last paragraph is erroneous. I must have had a typo in my testing. -- Neil Cerutti -- https://mail.python.org/mailman/listinfo/python-list
Re: converting letters to numbers
On Tue, Oct 8, 2013, at 11:44, kjaku...@gmail.com wrote:
def add(c1, c2):
ans = ''
This only makes sense if your answer is going to be multiple characters.
for i in c1 + c2:
This line concatenates the strings together.
ans += chrord(i)-65))%26) + 65)
The way you are doing the modulus, this results in - well, let me
illustrate:
add('','WXYZ[\]^_`abcde')
'WXYZABCDEFGHIJK'
--
https://mail.python.org/mailman/listinfo/python-list
Re: Re for Apache log file format
Aiui apache log format uses space as delimiter, encapsulates strings in '' characters, and uses '-' as an empty field. Specifying the field delimiter as a space, you might be able to use the csv module to read these. I haven't done any Apache log file work since long before the csv module was available, but it just might work. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Cookie gets changed when hit comes from a referrer
On Tue, 08 Oct 2013 13:04:34 +0300, Νίκος Αλεξόπουλος wrote: I use this code to retrive or set a cookie to the visitor's browser if present and identify him bu it. You are aware that using cookies to track a user who doesn't want to be tracked won't work, because he'll just tell his browser to not use cookies, aren't you. Nick, if a user doesn't want to be tracked, you can't track them. The user controls all the data their machine sends to you. This means that they can manipulate it. Nothing you can do will prevent this. -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: converting letters to numbers
On 08/10/2013 15:28, kjaku...@gmail.com wrote:
I have to define a function add(c1, c2), where c1 and c2 are capital letters;
the return value should be the sum (obtained by converting the letters to
numbers, adding mod 26, then converting back to a capital letter).
I'd say the requirement is lacking in that no encoding is specified.
All I have so far is:
def add(c1, c2):
ord(c1) - ord('a') + 1
ord(c2) - ord('a') + 1
I know I need to use ord and chr, just not sure how.
I'll further observe from your later replies that you're suffering from
the highly contagious, highly virulent double line spacing disease.
This is known to cause severe eye strain leading to blindness. In can
be cured by purchasing medication here
https://wiki.python.org/moin/GoogleGroupsPython
--
Roses are red,
Violets are blue,
Most poems rhyme,
But this one doesn't.
Mark Lawrence
--
https://mail.python.org/mailman/listinfo/python-list
Re: Cookie gets changed when hit comes from a referrer
Στις 8/10/2013 6:55 μμ, ο/η Denis McMahon έγραψε: On Tue, 08 Oct 2013 13:04:34 +0300, Νίκος Αλεξόπουλος wrote: I use this code to retrive or set a cookie to the visitor's browser if present and identify him bu it. You are aware that using cookies to track a user who doesn't want to be tracked won't work, because he'll just tell his browser to not use cookies, aren't you. Nick, if a user doesn't want to be tracked, you can't track them. The user controls all the data their machine sends to you. This means that they can manipulate it. Nothing you can do will prevent this. Yes iam aware of that, but its the best trcking method i can think of. Tracking just the hostname is not accurate since with every router restart, that info is changing. Tracking the visitor by settign a cookie to its browser is not perfect/accurate since he can manipulate its broswer data or flush the cookies but this is the best one can do after having people register on the webiste. Or perhaps trying to identify the cookie + hostname is even better. Can you help me with this particuler problem please? -- What is now proved was at first only imagined! WebHost http://superhost.gr -- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On 2013-10-08 15:36, Denis McMahon wrote: On Tue, 08 Oct 2013 08:33:48 -0400, Roy Smith wrote: In the old days, it used to be /usr/dict/words. Port Python to v6, and save another 6 characters :-) Doesn't matter where it is, a link to it exists at /w now ;) You prodigal...wasting a /. I just symlinked it from my current working directory so it exists at w. ;-) -tkc -- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On 2013-10-08 17:17, Chris Angelico wrote: Who's up for some fun? Implement an XKCD-936-compliant password generator in Python 3, in less code than this: print(*__import__(random).sample(open(/usr/share/dict/words).read().split(\n),4)) Second challenge: Use it for generating all your passwords :) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 9104: ordinal not in range(128) -tkc -- https://mail.python.org/mailman/listinfo/python-list
Re: class-private names and the Zen of Python
On 10/8/13 7:15 AM, Marco Buttu wrote: Also, as Terry mentions, no one has ever assigned the two names you show, Sincerely, I can not now if someone has assigned (or will assegne) in such way... If you explain more about what you are building, and where this crops up as a problem, we can help come up with a solution. --Ned. -- https://mail.python.org/mailman/listinfo/python-list
Re: Encoding of surrogate code points to UTF-8
On 08/10/2013 16:23, Pete Forman wrote:
Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes:
I think this is a bug in Python's UTF-8 handling, but I'm not sure.
[snip]
py s = '\ud800\udc01'
py s.encode('utf-8')
Traceback (most recent call last):
File stdin, line 1, in module
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in
position 0: surrogates not allowed
Have I misunderstood? I think that Python is being too strict about
rejecting surrogate code points. It should only reject lone surrogates,
or invalid pairs, not valid pairs. Have I misunderstood the Unicode FAQs,
or is this a bug in Python's handling of UTF-8?
http://www.unicode.org/versions/Unicode6.2.0/ch03.pdf
D75 Surrogate pair: A representation for a single abstract character
that consists of a sequence of two 16-bit code units, where the first
value of the pair is a high-surrogate code unit and the second value
is a low-surrogate code unit.
* Surrogate pairs are used only in UTF-16. (See Section 3.9, Unicode
EncodingForms.)
* Isolated surrogate code units have no interpretation on their own.
Certain other isolated code units in other encoding forms also have no
interpretation on their own. For example, the isolated byte [\x80] has
no interpretation in UTF-8; it can be used only as part of a multibyte
sequence. (See Table 3-7). It could be argued that this line by itself
should raise an error.
That first bullet indicates that it is indeed illegal to use surrogate
pairs in UTF-8 or UTF-32.
The only time you should get a surrogate pair in a Unicode string is in
a narrow build, which doesn't exist in Python 3.3 and later.
--
https://mail.python.org/mailman/listinfo/python-list
Re: parsing email from stdin
On 08.10.2013 17:25, Antoon Pardon wrote:
Op 08-10-13 16:24, Andreas Perstinger schreef:
Looking at the docs, I've found there is also message_from_binary_file
which works for me with your code.
http://docs.python.org/3/library/email.parser.html#email.message_from_binary_file
I can't try that out right now, but I had a look at the code and the
ByteParser that is mentioned their looks like this:
class BytesFeedParser(FeedParser):
Like FeedParser, but feed accepts bytes.
def feed(self, data):
super().feed(data.decode('ascii', 'surrogateescape'))
Somehow I doubt that trying to decode my utf-8 stream as if it was
ascii will work.
Actually it does work:
$ cat testmail.txt
From: someone some...@example.com
To: me m...@example.com
Subject: something
Content-Type: text/plain; charset=UTF-8;
Content-Transfer-Encoding: 8bit
foo bar AÄÖÜĎӅ baz
$ file testmail.txt
testmail.txt: news or mail, UTF-8 Unicode text
$ cat foo.py
#!/usr/bin/python3
import sys
from email import message_from_binary_file
sys.stdin = sys.stdin.detach()
msg = message_from_binary_file(sys.stdin)
print(from: , msg['From'])
print(to: , msg['To'])
print(subject: , msg['Subject'])
print(body: , msg.get_payload())
$ ./foo.py testmail.txt
from: someone some...@example.com
to: me m...@example.com
subject: something
body: foo bar AÄÖÜĎӅ baz
Bye, Andreas
--
https://mail.python.org/mailman/listinfo/python-list
Re: Cookie gets changed when hit comes from a referrer
Is there any better way to identif a previous visitor? i tried cookies
which failed for me for the reason i opened this thread and host like
follows:
# try to locate the visitor
cur.execute('''SELECT * FROM visitors WHERE counterID = %s and host =
%s''', (cID, host) )
data = cur.fetchone()
if not data:
# if first time visitor on this page, create new record
cur.execute('''INSERT INTO visitors (counterID, host, city, useros,
browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s)''',
(cID, host, city, useros,
browser, ref, lastvisit) )
else:
# since visitor exists just update his record
cur.execute('''UPDATE visitors SET city = %s, useros = %s, browser =
%s, ref = %s, hits = hits + 1, lastvisit = %s''', (city, useros,
browser, ref, lastvisit) )
===
Please tell me if you can think fo something else.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On 10/08/2013 09:07 AM, Tim Chase wrote: On 2013-10-08 15:36, Denis McMahon wrote: On Tue, 08 Oct 2013 08:33:48 -0400, Roy Smith wrote: In the old days, it used to be /usr/dict/words. Port Python to v6, and save another 6 characters :-) Doesn't matter where it is, a link to it exists at /w now ;) You prodigal...wasting a /. I just symlinked it from my current working directory so it exists at w. ;-) -tkc Yeah, but that's a lot of pixels! Link it to ' in the current directory. -- https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
I don't have an infinite stack to implement lambda calculus, but... And then But this is not a useful formalism. Any particular Program implements a DFA, even as it runs on a TM. The issue of whether than TM is finite or not can be dismissed because a simple calculation can usually suffice, or at least establish a range usefulness so as not to run out of memory. Having it both ways aren't you? I'm just speaking from programmer experience and the fact that most machines are VonNeumann architecture. Being that as it is, maxing out the stack simply happens, and I don't dare do any non-simple recursion, but otherwise, practically speaking, I can calculate my memory usage that may grow on the heap so that is effectively a non-issue. This may not be an important distinction for computing, the art (Hello ultimate lambda friends), but it is significant for the computing, the science. MarkJ Tacoma, Washington -- https://mail.python.org/mailman/listinfo/python-list
Re: Cookie gets changed when hit comes from a referrer
On Tue, Oct 8, 2013 at 1:30 PM, Νίκος Αλεξόπουλος nikos.gr...@gmail.com wrote:
Is there any better way to identif a previous visitor? i tried cookies which
failed for me for the reason i opened this thread and host like follows:
# try to locate the visitor
cur.execute('''SELECT * FROM visitors WHERE counterID = %s
and host = %s''', (cID, host) )
data = cur.fetchone()
if not data:
# if first time visitor on this page, create new
record
cur.execute('''INSERT INTO visitors (counterID,
host, city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s,
%s)''',
(cID, host, city, useros,
browser, ref, lastvisit) )
else:
# since visitor exists just update his record
cur.execute('''UPDATE visitors SET city = %s, useros
= %s, browser = %s, ref = %s, hits = hits + 1, lastvisit = %s''', (city,
useros, browser, ref, lastvisit) )
===
Please tell me if you can think fo something else.
Yes! there is a very simple and comprehensive way to learn about your
visitors. Use Google Analytics. Its free, you only need a google
account to open an analytics account. They give you a small bit of
javascript that you copy and past to your pages. If you are using a
template to create your pages, this is easy, since you just add google
code to the template.
--
https://mail.python.org/mailman/listinfo/python-list
--
Joel Goldstick
http://joelgoldstick.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On 10/07/2013 11:17 PM, Chris Angelico wrote: Who's up for some fun? Implement an XKCD-936-compliant password generator in Python 3, in less code than this: print(*__import__(random).sample(open(/usr/share/dict/words).read().split(\n),4)) Second challenge: Use it for generating all your passwords :) [1] https://en.wikipedia.org/wiki/Code_golf [2] http://xkcd.com/936/ ChrisA So how about finding the last word that starts with each lower case letter of the alphabet in turn: azures bywords czars ... Tobiah -- https://mail.python.org/mailman/listinfo/python-list
Re: Encoding of surrogate code points to UTF-8
sys.version
'3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600 32 bit (Intel)]'
'\ud800'.encode('utf-8')
Traceback (most recent call last):
File eta last command, line 1, in module
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in position
0:
surrogates not allowed
'\ud800'.encode('utf-32-be')
b'\x00\x00\xd8\x00'
'\ud800'.encode('utf-32-le')
b'\x00\xd8\x00\x00'
'\ud800'.encode('utf-32')
b'\xff\xfe\x00\x00\x00\xd8\x00\x00'
jmf
--
https://mail.python.org/mailman/listinfo/python-list
Re: Cookie gets changed when hit comes from a referrer
On Tue, Oct 8, 2013 at 12:04 PM, Νίκος Αλεξόπουλος nikos.gr...@gmail.com wrote: Στις 8/10/2013 6:55 μμ, ο/η Denis McMahon έγραψε: On Tue, 08 Oct 2013 13:04:34 +0300, Νίκος Αλεξόπουλος wrote: I use this code to retrive or set a cookie to the visitor's browser if present and identify him bu it. Browser cookies have been defined and around for a very long time. If you google browser cookie tutorial you can learn how they work -- probably within an hour!. This will help you find your solution The first poster pointed out that www.example.com and example.com can be considered different domains. You can make a cookie work for both but you need to understand cookies to learn how. This is off topic, ... again! https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick http://joelgoldstick.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Cookie gets changed when hit comes from a referrer
On Tue, 08 Oct 2013 19:04:37 +0300, Νίκος Αλεξόπουλος wrote: Can you help me with this particuler problem please? Unfortunately I can't, because I am unable to reproduce the problem you describe. When I load my test page in the browser, then replace it with something else by entering an address in the address bar and pressing return, then use the back link followed by the reload one, I am back at my test page with the original cookie value. Of course, this is using my cookie etc code and mechanisms, and not yours Now, either it's an issue in your python implementation of cookie handling which isn't happening in my implementation, or it's something to do with the way that your system passes data around (cgi) that doesn't happen in mine (mod_wsgi), or it's happening in the browser you're testing in, but not in my browser. Have you checked the cookie jar in the browser to see what value the cookie has? Is that the value you think it should have? Note that checking the cookie jar is a browser topic, not a python topic, so if you don't know how to do that you're going to have to find the right place to ask, WHICH IS NOT HERE! Ideally you need to check what the server thinks it's setting the cooking to, what the browser thinks it received as the cookie, and what the server gets back afterwards to work out where the error is happening. -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On Tuesday, October 8, 2013 5:47:56 AM UTC-10, rand...@fastmail.us wrote: Importing random as r doesn't actually save anything, since as r is the same five characters you saved from the one use of it. I realize, it just looks nicer than the original __import__, and since it doesn't add any characters... The optimization was using readlines. -- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On Wed, Oct 9, 2013 at 7:27 AM, sprucebond...@gmail.com wrote: On Tuesday, October 8, 2013 5:47:56 AM UTC-10, rand...@fastmail.us wrote: Importing random as r doesn't actually save anything, since as r is the same five characters you saved from the one use of it. I realize, it just looks nicer than the original __import__, and since it doesn't add any characters... The optimization was using readlines. Are you aware that readlines keeps the \n at the end of each line, though? Looks a lot less clean in its output that way. That's why I used .split() in the first place. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
How to help us to help you (was: Image manipulation)
markot...@gmail.com writes: Well english isnt my native language, and there are things i just dont know how to explain in any language. I sympathise with attempting to explain things in a foreign language. Sorry that you have that difficulty here. But the rest of Steven's advice is sound: you need to help us to help you. One way to do that: Please don't use Google Groups to post here, it mangles your replies in ways that make it difficult for you to be understood URL:https://wiki.python.org/moin/GoogleGroupsPython. And i cant give all the codes i am using, since there are loads of pieces i do not own, but i know the people who made them. So you need to present examples that are short, self-contained, correct, compilable URL:http://sscce.org/. That may entail you writing the examples for the purpose of posting them here; but do run them yourself before posting! You'll often learn about the problem without needing our input if you do it that way. And if you don't, then you have a tool that will definitely get you better responses here. -- \“The problem with television is that the people must sit and | `\keep their eyes glued on a screen: the average American family | _o__) hasn't time for it.” —_The New York Times_, 1939 | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: converting letters to numbers
On 8/10/2013 10:28, kjaku...@gmail.com wrote:
I have to define a function add(c1, c2), where c1 and c2 are capital letters;
the return value should be the sum (obtained by converting the letters to
numbers, adding mod 26, then converting back to a capital letter).
All I have so far is:
def add(c1, c2):
ord(c1) - ord('a') + 1
ord(c2) - ord('a') + 1
I know I need to use ord and chr, just not sure how.
Factor the problem into three functions. one function converts a
one-character string into an int, or gives an exception if the
character. isn't uppercase ASCII.
Second function converts a small int into a string containing one
uppercase ASCII letter, throwing an exception if negative or above 25.
Third function takes two string arguents, throws an exception if either
of them is not exactly one character in length. Then it calls the first
function twice, adds the results, modulos it, and calls the second
function, returning its return value.
Which of these is giving you trouble? Notice you can use the first two
functions to test each other.
--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
On Wed, Oct 9, 2013 at 12:11 AM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Tue, 08 Oct 2013 18:16:01 +0530, Ravi Sahni wrote: So in that sense, computers are Turing Machines. Anything a physical computing device can compute, a Turing Machine could too. The converse is not true though: a Turing Machine with infinite tape can compute things where a real physical device would run out of memory, although it might take longer than anyone is willing to wait. Thanks Sir the detailed explanation. You are offering me many thoughts inside few words so I will need some time to meditate upon the same. Presently Sir, I wish to ask single question: What you mean wave our hands?? It is an idiom very common in Australia. (It may not be well known in the rest of the English-speaking world.) It means to figuratively flap one's hands around in the air while skipping over technical details or complications. For example, we often talk about hand-wavy estimates for how long a job will take: my hand-wavy estimate is it will take two days is little better than a guess. A derivative of the term has gone mainstream, too: http://tvtropes.org/pmwiki/pmwiki.php/Main/HandWave The term is commonly used when moving to a higher level of abstraction - we all know a computer doesn't have a soul, can't feel, and is ultimately just executing code and crunching numbers, but we handwave that (eg) the computer thought that this program was a risk, and that's why it quarantined it. When you're trying to explain to some user that he can't email .EXE files around, it's easier to take the slightly-inaccurate but simple explanation, hence the handwaves. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Encoding of surrogate code points to UTF-8
On 10/8/2013 9:52 AM, Steven D'Aprano wrote:
I think this is a bug in Python's UTF-8 handling, but I'm not sure.
If I've read the Unicode FAQs correctly, you cannot encode *lone*
surrogate code points into UTF-8:
http://www.unicode.org/faq/utf_bom.html#utf8-5
Sure enough, using Python 3.3:
py surr = '\udc80'
I am pretty sure that if Python were being strict, that would raise an
error, as the result is not a valid unicode string. Allowing the above
or not was debated and laxness was allowed for at least the following
practical reasons.
1. Python itself uses the invalid surrogate codepoints for
surrogateescape error-handling.
http://www.python.org/dev/peps/pep-0383/
2. Invalid strings are needed for tests ;-)
-- like the one you do next.
3. Invalid strings may be needed for interfacing with other C APIs.
py surr.encode('utf-8')
Traceback (most recent call last):
File stdin, line 1, in module
UnicodeEncodeError: 'utf-8' codec can't encode character '\udc80' in
position 0: surrogates not allowed
Default strict encoding (utf-8 or otherwise) will only encode valid
unicode strings. Encode invalid strings with surrogate codepoints with
surrogateescape error handling.
But reading the previous entry in the FAQs:
http://www.unicode.org/faq/utf_bom.html#utf8-4
I interpret this as meaning that I should be able to encode valid pairs
of surrogates.
It says you should be able to 'convert' them, and that the result for
utf-8 encoding must be a single 4-bytes code for the corresponding
supplementary codepoint.
So if I find a code point that encodes to a surrogate pair
in UTF-16:
py c = '\N{LINEAR B SYLLABLE B038 E}'
py surr_pair = c.encode('utf-16be')
py print(surr_pair)
b'\xd8\x00\xdc\x01'
and then use those same values as the code points, I ought to be able to
encode to UTF-8, as if it were the same \N{LINEAR B SYLLABLE B038 E} code
point. But I can't:
py s = '\ud800\udc01'
This is now a string with two invalid codepoints instead of one ;-).
As above, it would be rejected if Python were being strict.
py s.encode('utf-8')
Traceback (most recent call last):
File stdin, line 1, in module
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in
position 0: surrogates not allowed
Have I misunderstood? I think that Python is being too strict about
rejecting surrogate code points.
No, it is being too lax about allowing them at all.
I believe there is an issue on the tracker (maybe closed) about the doc
for unicode escapes in string literals. Perhaps is should say more
clearly that inserting surrogates is allowed but results in an invalid
string that cannot be normally encoded.
--
Terry Jan Reedy
--
https://mail.python.org/mailman/listinfo/python-list
