Problem pickling exceptions in Python 2.5/2.6
I'm having troubles pickling classes that extend Exception. Given the following source: class Foo(object): def __init__(self, m): self.m=m class Bar(Exception): def __init__(self, m): self.m=m import pickle s=pickle.dumps(Foo(test)) pickle.loads(s) # normal object works s=pickle.dumps(Bar(test)) pickle.loads(s) # exception object fails When running this in Python 2.5 or Python 2.6a, it crashes on the last line, with: [C:\]c:\python25\python.exe pickling.py Traceback (most recent call last): File pickling.py, line 14, in module pickle.loads(s) # exception object fails File C:\Python25\lib\pickle.py, line 1374, in loads return Unpickler(file).load() File C:\Python25\lib\pickle.py, line 858, in load dispatch[key](self) File C:\Python25\lib\pickle.py, line 1133, in load_reduc value = func(*args) TypeError: __init__() takes exactly 2 arguments (1 given) This used to work fine in Python 2.3 and 2.4... What has been changed in 2.5? Is there a way to get rid of the error? (I tried some stuff with __getinitargs__ and __getnewargs__ but couldn't get it to work). --irmen -- http://mail.python.org/mailman/listinfo/python-list
[issue2871] store thread.get_ident() thread identifier inside threading.Thread objects
Irmen de Jong [EMAIL PROTECTED] added the comment: Thanks Gregory, for taking the time to make a patch. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2871 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2871] store thread.get_ident() thread identifier inside threading.Thread objects
Irmen de Jong [EMAIL PROTECTED] added the comment: Adding it in the run method would only work for threads that I create in my own code. The thing is: I need to be able to get the tread identification from threads created by third party code. So I cannot rely on that code putting it in the thread object themselves like that. Hence my wish of letting the standard library module take care of it. And using the id() of the current thread object has a rather obscure problem. I was using it as a matter of fact, until people reported problems in my code when used with certain atexit handling. (Sometimes the wrong id() is returned). Because of that I wanted to switch to the more low-level thread.get_ident() identification of different threads, because that is supposed to return a stable os-level thread identification, right? __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2871 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: TPCServer and xdrlib
Laszlo Nagy wrote: It is possible to change the serialization used by Pyro http://pyro.sourceforge.net/manual/9-security.html#pickle to the the 'gnosis' XML Pickler. As I said earlier, I would not use XML. Just an example - I need to be able to transfer image files, word and excel documents. How silly it would be to base64encode a binary file, then put it into an XML. L Fair enough. In that case, here's 5 suggestions: - use simple file copying from a mounted network drive - use http (web server) - use ftp - use scp - use rsync Why wouldn't one of these work for you? Did I miss something in your original requirements? All of the above high level protocols are very efficient in concurrently transferring files from a server to multiple clients. --irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie: Keep TCP socket open
Alan Wright wrote: while (num1=10) : s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.settimeout(10.0) s.connect((10.1.1.69, 50008)) # SMTP print s.recv(1024) + '\n', num1=num1+1 #s.close() sys.exit(1) I think the following is happening: Reusing the 's' object for every new socket will make Python to garbage collect the previous ones. Garbage collecting a socket will likely close() it. Also after creating all sockets your program exits. I guess either Python or the operating system itself will go close all the sockets. Try putting every new socket you make into a big list instead, so that Python can't garbage collect it. And put your program to sleep at the end. import time allsockets=[] while (...): s=socket.socket(... allsockets.append(s) s.settimeout(... ... time.sleep(9) --irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: How do *you* use Python in non-GUI work?
John Salerno wrote: Hey all. Just thought I'd ask a general question for my own interest. Every time I think of something I might do in Python, it usually involves creating a GUI interface, so I was wondering what kind of work you all do with Python that does *not* involve any GUI work. This could be any little scripts you write for your own benefit, or what you do at work, if you feel like talking about that! :) Thanks. - web server/blog/wiki. This doesn't have a gui if you don't consider HTML pages a gui - various scripts to help me during software deployment and testing (for instance log file analyzers, automated web clients) - network communications library that tries hard to just get out of your way ;-) - file processing scripts to do more sophisticated stuff than basic search/replace - most recent script is a little tool that downloads the latest version of some World-Of-Warcraft addon, extracts it to the game folder after deleting the old one first, and then copies a backup to a network drive. I just doubleclick the .py file and it dumps the results in a console window that closes after a few seconds. Who needs a gui for that? Also, I often find myself opening a Python prompt to just execute simple tasks that I see other people needing big tools or even online services for: - base-64 encoding/decoding - md5/sha hashing - simple string or regular expression operations - simple math - unicode decoding/encoding - etc etc. --irmen de jong -- http://mail.python.org/mailman/listinfo/python-list
[issue2871] store thread.get_ident() thread identifier inside threading.Thread objects
New submission from Irmen de Jong [EMAIL PROTECTED]: I've ran into a problem where it would be very nice to be able to tell the tread.get_ident() of a given threading.Thread object. Currently, when creating a new Thread object, there is no good way of getting that thread's get_ident() value. I propose adding the get_ident() value as a publicly accessible field of every threading.Thread object. -- components: Extension Modules messages: 66882 nosy: irmen severity: normal status: open title: store thread.get_ident() thread identifier inside threading.Thread objects type: feature request __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2871 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Best technology for agent/web server architecture
Gabriel Genellina wrote: 2008/5/8 M.-A. Lemburg [EMAIL PROTECTED]: SOAP would be a good choice if you want to send to data to other servers as well, e.g. Java-based ones. XML-RPC and JSON are better for simple data structures. If you have control over both client and server and don't need to bother with other backends or frontends, Python pickle is the best choice. En Fri, 09 May 2008 05:41:07 -0300, Florencio Cano [EMAIL PROTECTED] escribió: I have control over agent and client but I'm not sure how to use pickle for this task. Do you suggest to pickle the objects that I want to send and send it over a usual socket? I have searched a bit in Google and I have seen that Pickle is insecure by default. What do you think about this? insecure means that someone could build a specially crafted pickle able to run arbitrary code on the unpickling environment. One way to avoid that is to only accept pickles from trusted sources: using SSL by example. While Pyro (http://pyro.sourceforge.net) uses pickle by default, it is well understood that you'll have to deal with a potential security issue if your server is open to untrusted/uncontrolled clients. Pyro provides several things that could help you here: - you can define a connection authenticator that checks client IP and/or passphrases - you can switch to an XML based serialisation protocol (courtesy of gnosis tools) - you can run Pyro over SSL and let SSL deal with authentication/encryption/... Cheers Irmen de Jong -- http://mail.python.org/mailman/listinfo/python-list
Re: Import/Create module from buffer
Gruik wrote: But before that 1 question: what if I'm in Python ? Following your solution, I did that in Python : def load_buffer(buffer) : compiled_buffer = compile(buffer, module_name, exec) exec(compiled_buffer) It works great except that I can't have a module object and that it is as if I did from module import * But I need the module object and not an import * behavior. Any idea about the way to do that? Something along the lines of: import new mymodule = new.module(mymodule) exec code in mymodule.__dict__ --irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: what does int means for python ?
alefajnie wrote: #!/usr/bin/env python arr = [[0 for c in range(0, 10)] for r in range(0, 10)] # 10x10 array arr[2,3] # throws TypeError: list indices must be integers arr[int(2), int(3)] # also throws TypeError: list indices must be integers ! - what is wrong in above script ? try arr[2][3] instead. basically your version is trying to index an array with a tuple argument (2,3). --irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple question
Gandalf wrote: On May 10, 2:36 pm, Bjoern Schliessmann usenet- [EMAIL PROTECTED] wrote: Gandalf wrote: how can i ran script with python It depends on your web server configuration. To get your web server execute Python code, there are several alternatives like * CGI * FastCGI * mod_python Regards, Björn -- BOFH excuse #93: Feature not yet implemented my server is my computer and all i did way to install python on it. You might want to read a bit more about Python and how it can be used for web programming. Because your question is not easily answered. I suggest the following at least: http://wiki.python.org/moin/PythonVsPhp (especially the Compared as Web Development Frameworks section) http://wiki.python.org/moin/WebProgramming Good luck. --irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Receive data from socket stream
[EMAIL PROTECTED] wrote: Until now, I've been doing this little trick: data = client.recv(256) new = data while len(new) == 256: new = client.recv(256) data += new Are you aware that recv() will not always return the amount of bytes asked for? (send() is similar; it doesn't guarantee that the full buffer you pass to it will be sent at once) I suggest reading this: http://www.amk.ca/python/howto/sockets/sockets.html --irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Using pickle for interprocess communication - some notes and things that ought to be documented.
Christian Heimes wrote: John Nagle wrote: It's possible to use pickle for interprocess communication over pipes, but it's not straightforward. IIRC the processing module uses pickle for IPC. Maybe you can get some idea by reading its code? http://pypi.python.org/pypi/processing/0.40 Christian So does Pyro: http://pyro.sourceforge.net/ However Pyro uses TCP-IP sockets for communication. It uses a small header that contains the size of the message and a few other things, and then the (binary by default) pickle stream. --irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: RMI with Pyro et al
Sells, Fred wrote: Diez B. Roggisch wrote . Why do you want that (hot deploy) anyway? Does startuptime of a script really bother you? shouldn't take more than a few seconds. My primary need is development/debug. I'm a Pyro newbie and I add a feature and then test. The only way I've found to kill the Pyro server on XP is to open the task manager and kill the python task (hopefully the right one). How do you start the pyro server? Are you talking about the Windows Services that you can simply start/stop with: net start PyroNS net start PyroES net stop PyroNS net stop PyroES ?? Often it is way simpler to just use the pyro-ns.cmd / pyro-es.cmd batch files and stop the server with a simple ^C in the console window. -irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: RMI with Pyro et al
Diez B. Roggisch wrote: Go install cygwin (but not it's included python-interpreter, or at least make sure you have your python path properly under control) and then simply start the script from the command-line. And hit C-c if you need it to stop, and restart it. Only start it as service if it's deployed. What's cygwin got to do with it? -irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: Compyler 0.1
Thanks Grant for the very informative response. -irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: Compyler 0.1
Grant Olson wrote: Compyler is a pre-alpha x86 native code compiler. In what ways is this similar or different to Shed Skin? http://mark.dufour.googlepages.com/ --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: question: howto transfer objects between server and client?
OpenPavilion wrote: Since XMLRPC has limited features: Is there any other server/client technique to transfer objects (not strings, or dictionaries, but self defined object types) ? Take a look at Pyro; http://pyro.sourceforge.net --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: distutils
[EMAIL PROTECTED] wrote: Hello again, Is there any patch for python distutils, for this from distutils import log,dir_util ImportError: cannot import name log Regards, Vedran Are you sure you haven't written a module yourself called distutils.py ? (that one will hide the standard distutils module) --irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Auto run/Timer
Rohan wrote: Hello, I would like my script to run once a week with out any external interference. More like a timer. Can it be done in python or should some other shell scripting be used. If anyone knows anything please let me know. Have a look at my 'kronos' task scheduler, available from: http://www.razorvine.net/downloads.html Things like this do require a Python process to be running all the time, for obvious reasons. If you don't want that, you'll have to use a task scheduler tool that your operating system provides (cron, for instance). --irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: (no) fast boolean evaluation ?
John Machin wrote: (you_are_confused and/or function_returns_bool_but_has__side_effects()) That above expression should be written more explicitly like: function_result = function_returning_bool_but_with_side_effects() if you_are_confused or function_result: do_something_nice() --irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Directory
Rohan wrote: I would like to get a list of sub directories in a directory. If I use os.listdir i get a list of directories and files in that . i only want the list of directories in a directory and not the files in it. anyone has an idea regarding this. Look up os.walk (allows you to selectively walk over files and directories) and/or os.path.isdir (allows you to test if a given path is a directory) --irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Pickled objects over the network
Rustom Mody wrote: Sure pyro may be the solution but it may also be overkill Why not use safe_load from the yaml module? In what way would Pyro be overkill where Yaml (also a module that you need to install separately) wouldn't be? -irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Best method for inter process communications
JamesHoward wrote: I am looking for a way of performing inter process communication over XML between a python program and something else creating XML data. What is that something else? --irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: socket: connection reset by server before client gets response
ahlongxp wrote: me again. Connection reset by peer happens about one in fifth. I'm using python 2.5.1 and ubuntu 7.04. -- ahlongxp Software College,Northeastern University,China [EMAIL PROTECTED]://www.herofit.cn Post the code. Without it we can only help when our magic crystal balls are back from service. ==irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Python and ARexx (was Re: Tiny/small/minimalist Python?)
Dennis Lee Bieber wrote: On 3 Jul 2007 10:03:45 GMT, Jorgen Grahn [EMAIL PROTECTED] declaimed the following in comp.lang.python: How does Python combine with ARexx? Can you control applications which provide an ARexx interface? Irmen had supplied a Python module that had ARexx port operations. It was not only possible to control a program with an ARexx port from your Python programs, but it was also possible to create an ARexx host program in Python. That is: a program exposing an ARexx command port to receive ARexx messages. This could be done in a few lines of (python) code. If you're interested, a bit more info is on the AmigaPython page: http://www.monkeyhouse.eclipse.co.uk/amiga/python/ --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Tiny/small/minimalist Python?
Paul Rubin wrote: rtk [EMAIL PROTECTED] writes: FYI.. I wanted a simple version of Python to run on an ancient DEC Alpha box. I got VMS Python 2.5 up and running but it is too slow to use. It takes *minutes* to get the interpreter prompt after typing 'python'! Something is wrong. Maybe it's trying to DNS itself and timing out, or something like that. Something is definately wrong. Back in the days my port of Python to the Commodore Amiga machine ran quite comfortably on a 50 mhz CPU with 4 Mb of RAM. (ok ok it was Python 1.5.2, that has to be said). Python started in about 5 seconds on that Amiga if I remember correctly. I'm quite sure your 'ancient' DEC Alpha box is way more powerful than my Amiga back then. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Periodic tasks.
Ramashish Baranwal wrote: Hi, I am trying to execute some tasks periodically, those familiar with unix can think of it as equivalent to cron jobs. I have tried looking around, but couldn't find a way. Would appreciate any pointers or clues.. Thanks, -Ram Have a look at Kronos, a simple task scheduler I wrote a while ago, based on sched. It's part of Turbogears as well: http://trac.turbogears.org/browser/trunk/turbogears/scheduler.py --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Shared Memory Space - Accross Apps Network
Hendrik van Rooyen wrote: Just to get the ball rolling, I'd suggest two things: Pyro -http://pyro.sf.net This is good advice, if you have the power to run it. What do you mean exactly by the power to run it? --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Components for a client/server architecture
John Nagle wrote: You don't hear much about CORBA any more. It used to be derided as a bulky way to marshall data, but then came XML. CORBA is much more than just a way to marshall data. GIOP (or its more often used implementation IIOP) is the marshaling protocolused in CORBA. And it is hardly bulky! Being a binary protocol, it is much faster and leaner than XML. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Pyro 3.7 (remote objects)
I'm happy to announce Pyro 3.7 -- Python's own powerful remote method invocation technology! You can get it via http://pyro.sourceforge.net, then go to the SF project homepage download area. This is a small improvement release since Pyro 3.6. New stuff: - bdist_rpm typo fix in setup.cfg - renamed all batch files with 'pyro-' prefix to avoid name clashes (debian package already had this) - NS broadcast retries are a bit faster now - Pyro.core.SynchronizedObjBase now correctly handles string exceptions - the NS nt service won't respond to shutdown requests anymore - wxnsc updated to recent WxPython API, deprecation warning is gone Have fun, and thanks for your interest, support, and feedback! --Irmen de Jong --- What is Pyro? Pyro is an acronym for PYthon Remote Objects. Pyro is an advanced and powerful Distributed Object Technology system written entirely in Python, that is designed to be very easy to use. It is extremely easy to implement a distributed system with Pyro, because all network communication code is abstracted and hidden from your application. You just get a remote Python object and invoke methods on the object on the other machine. Pyro offers you a Name Server, an Event Service, mobile objects, remote exceptions, dynamic proxies, remote attribute access, automatic reconnection, a very good and detailed manual, and many examples to get you started right away. -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
ANN: Pyro 3.7 (remote objects)
I'm happy to announce Pyro 3.7 -- Python's own powerful remote method invocation technology! You can get it via http://pyro.sourceforge.net, then go to the SF project homepage download area. This is a small improvement release since Pyro 3.6. New stuff: - bdist_rpm typo fix in setup.cfg - renamed all batch files with 'pyro-' prefix to avoid name clashes (debian package already had this) - NS broadcast retries are a bit faster now - Pyro.core.SynchronizedObjBase now correctly handles string exceptions - the NS nt service won't respond to shutdown requests anymore - wxnsc updated to recent WxPython API, deprecation warning is gone Have fun, and thanks for your interest, support, and feedback! --Irmen de Jong --- What is Pyro? Pyro is an acronym for PYthon Remote Objects. Pyro is an advanced and powerful Distributed Object Technology system written entirely in Python, that is designed to be very easy to use. It is extremely easy to implement a distributed system with Pyro, because all network communication code is abstracted and hidden from your application. You just get a remote Python object and invoke methods on the object on the other machine. Pyro offers you a Name Server, an Event Service, mobile objects, remote exceptions, dynamic proxies, remote attribute access, automatic reconnection, a very good and detailed manual, and many examples to get you started right away. -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with socket.recv()
xreload wrote: Hello ! So, lets do : sock.py http://forums.childrenwithdiabetes.com/showthread.php?t=5030; - it not ok , only some part of document. wget http://forums.childrenwithdiabetes.com/showthread.php?t=5030; - it ok ! sock.py http://www.google.com/; - it ok ! Why i got only some part of document ? This is some bug in sockets module or i do something wrong in my code? You have a bug in your code: def get_body(self): body = self.response.split(\r\n\r\n, 2) try: return body[1] except: return self.response The split is breaking up the response in 2 splits, that is, three parts. You meant to have it split up in TWO parts. So change the argument 2 to the split call, to 1. Also, why not just use httplib/urllib etc? Just curious. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Power Point Slides
Krypto wrote: Hi, I want to give a short presentation in my group about benefits of python, why should one use python and some basic starting information about python, about its data type, etc. Can somebody point me to the right references on the web. I have searched a lot and I do get quite a few but I thought to check on newsgroup for a better presentation, or if you have prepared any, can you please share it. Thanks Ahem, if you can't create one yourself, how could you ever give a convincing presentation about these subjects? You'll have to tell your own story... You know the benefits of Python etc. yourself, don't you? --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: getmtime differs between Py2.5 and Py2.4
Leo Kislov wrote: Let me guess: your E drive uses FAT filesystem? -- Leo Nope, its all NTFS on my system. Anyway this doesn't matter, as the true cause is explained in another reply in this thread (bug in c runtime library of Python 2.4). --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: getmtime differs between Py2.5 and Py2.4
Martin v. Löwis wrote:
Is this a bug?
Why don't you read the responses posted earlier? John Machin
replied (in [EMAIL PROTECTED])
that you are mistaken: There is NO difference between the outcome
of os.path.getmtime between Py2.5 and Py2.4. It always did return
UTC, and always will.
Regards,
Martin
Err.:
[E:\Projects]dir *.py
Volume in drive E is Data Serial number is 2C4F:9C2D
Directory of E:\Projects\*.py
31-03-2007 20:46 511 log.py
25-11-2006 16:59 390 p64.py
7-03-2007 23:07 207 sock.py
3-02-2007 16:15 436 threads.py
1.544 bytes in 4 files and 0 dirs16.384 bytes allocated
287.555.584 bytes free
[E:\Projects]c:\Python24\python.exe -c import os; print
os.path.getmtime('p64.py')
1164470381
[E:\Projects]c:\Python25\python.exe -c import os; print
os.path.getmtime('p64.py')
1164466781.28
This is python 2.4.4 and Python 2.5.1 on windows XP.
The reported time clearly differs.
--Irmen
--
http://mail.python.org/mailman/listinfo/python-list
Re: Numbers and truth values
Szabolcs wrote: Newbie question: Why is 1 == True and 2 == True (even though 1 != 2), but 'x' != True (even though if 'x': works)? Please check before you post: [E:\Projects]python Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. 2==True False Anyway, it helps to think about value domains (in your examples, numbers and strings) as having a single null value, and non-null values. The null value is considered False. The non-null values are considered True. The null value for numbers is 0 obviously, and for strings it is '' (the empty string). Non-zero numbers and non-empty strings are considered True when used in a boolean expression. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Select weirdness
Ron Garret wrote: I don't understand why socketserver calling select should matter. (And BTW, there are no calls to select in SocketServer.py. I'm using Python2.5.) You don't *need* a select at all. Socketserver just blocks on accept() and dispatches a handler on the new connection. Anyway, try the following instead: That won't work for POST requests. Why not? Just add some more code to deal with the POST request body. There should be a content-length header to tell you how many bytes to read after the header section has finished. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Select weirdness
Ron Garret wrote:
Here's my code. It's a teeny weeny little HTTP server. (I'm not really
trying to reinvent the wheel here. What I'm really doing is writing a
dispatching proxy server, but this is the shortest way to illustrate the
problem I'm having):
from SocketServer import *
from socket import *
from select import select
class myHandler(StreamRequestHandler):
def handle(self):
print ''
while 1:
sl = select([self.rfile],[],[])[0]
if sl:
l = self.rfile.readline()
if len(l)3: break
print l,
pass
pass
printself.wfile, 'HTTP/1.0 200 OK'
printself.wfile, 'Content-type: text/plain'
printself.wfile
printself.wfile, 'foo'
self.rfile.close()
self.wfile.close()
print ''
pass
pass
You shouldn't use a select() of your own inside the handle method.
The socketserver takes care of that for you.
What you need to do is read the input from the socket until you've reached
the end-of-input marker. That marker is an empty line containing just \r\n
when dealing with HTTP GET requests.
Any reason don't want to use the BasicHTTPServer that comes with Python?
Anyway, try the following instead:
from SocketServer import *
from socket import *
from select import select
class myHandler(StreamRequestHandler):
def handle(self):
print ''
while True:
l = self.rfile.readline()
print repr(l)
if not l or l=='\r\n':
break
printself.wfile, 'HTTP/1.0 200 OK'
printself.wfile, 'Content-type: text/plain'
printself.wfile
printself.wfile, 'foo'
self.rfile.close()
self.wfile.close()
print ''
def main():
server = TCPServer(('',8080), myHandler)
server.serve_forever()
if __name__ == '__main__': main()
--Irmen
--
http://mail.python.org/mailman/listinfo/python-list
Re: how to transfer integer on socket?
Frank Potter wrote: Is there any easy way to transfer 4 bit integer on socket? I think you mean a 4-byte integer? Look at the struct module, anyway. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: [offtopic?] problem with UDP broadcast on Windows XP
Hendrik van Rooyen wrote:
I am not sure if this is at all relevant - but I seem to recall seeing
something once that had a list of socket numbers, splitting them
between UDP TCP - can the socket actually rx UDP?
Yeah, as I wrote: when I'm sending UDP packets to the port directly
on the server's IP address, it responds just fine.
It's just the broadcast packets that don't seem to arrive.
(sent to ('broadcast',9090) )
Steve Holden wrote:
It's most likely, I suspect without knowing to much about it, that the
service is stalling because of a failure to pump Windows messages.
Irmen, are you taking any action in your service to ignore Windows
messages that your service process receives?
Hm, seeing that it processes TCP and directed UDP packets just fine,
there shouldn't be a problem here?
No I'm not knowingly doing stuff that ignores windows messages...
(I could maybe put up the code if someone wants to take a look but
not right now. Need to rip a fair deal out - there's a lot of
other stuff in there that's not relevant to the problem.)
--Irmen
--
http://mail.python.org/mailman/listinfo/python-list
Re: problem with UDP broadcast on Windows XP
Paul McGuire wrote: I would investigate Windows security settings as a likely culprit. My guess is that you are running WinXP SP2 with the default security policies, which are likely to prohibit such promiscuous behavior. Here's a URL that may shed some light, it seems surprisingly instructive for MS support: http://support.microsoft.com/kb/842242 - Some programs seem to stop working after you install Windows XP Service Pack 2 Paul, that was terrific. Seems that the windows firewall blocks the broadcast stuff for services. After I disabled the thing, my service works again as intended! Have to add this to the notes of my software ;) Wonderful that my windows specific problem got solved in this Python group. Thanks again for all the ideas everyone for my silly offtopic problem. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
[offtopic?] problem with UDP broadcast on Windows XP
Hello
Sorry this might be a bit offtopic but I don't really know where else
to post this question. If you could point me in the right direction that
is much appreciated.
I'm running into a weird thing on Windows XP.
I'm using Python 2.5 with latest pywin32 from Mark Hammond.
I have this UDP socket server that binds on ('',9090) and is used
to be a broadcast responder. It works fine when I start the server
from the cmd prompt. UDP broadcast packets sent to ('broadcast',9090)
arrive in the server.
However now I'm running the same server as a Windows NT Service.
The same code is executed, the UDP server socket is bound to the
same address. But this time, the UDP broadcast packets do NOT arrive !?
When sending an UDP packet to the server's IP address directly,
it works. So the socket server is running.
Only thing is it doesn't seem to receive any broadcast packets.
Is this a known 'feature' of a windows NT service?
Doesn't windows allow a service to receive broadcast packets?
Again, sorry that I post this here it is probably more of a windows
question. But I don't know a good place to ask this type of thing.
Thanks for any help,
--Irmen de Jong
--
http://mail.python.org/mailman/listinfo/python-list
Re: [offtopic?] problem with UDP broadcast on Windows XP
Gabriel Genellina wrote: Try running the service impersonating another user (not LOCAL_SERVICE, the default). You can change that from the service control panel. Alas, that didn't change anything. I made it run as a user account that has admin privileges even, and it still doesn't respond to the broadcasts. :-( --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: [offtopic?] problem with UDP broadcast on Windows XP
Gabriel Genellina wrote: Ouch, no more ideas from me. You'll surely get more answers from a Windows networking group - this appears not to be related to Python anyway. Yeah I know that... That's what I mentioned in my original post... But I'm a noob on that type of thing, no idea where to get reliable help. So I hoped the good folks in this group could help me out a little. :) Thanks for your idea anyway! -Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: low level networking in python
Maxim Veksler wrote:
I'm trying to bind a non-blocking socket, here is my code:
#!/usr/bin/env python
import socket, select
from time import sleep
s_nb1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s_nb1.setblocking(0)
s_nb1.bind(('192.168.2.106', 10002))
s_nb1.listen(5)
while 1:
conn, addr = s_nb1.accept()
ready_to_read, ready_to_write, in_error = select.select([conn], [],
[], 0)
print (ready_to_read, ready_to_write, in_error)
sleep(100)
s_nb1.close()
And this is the exception I'm getting:
python non_blocking_socket.py
Traceback (most recent call last):
File non_blocking_socket.py, line 13, in ?
conn, addr = s_nb1.accept()
File /usr/lib/python2.4/socket.py, line 161, in accept
sock, addr = self._sock.accept()
socket.error: (11, 'Resource temporarily unavailable')
What am I doing wrong here?
Nothing.
Any operation on a non-blocking socket that is usually blocking
(this includes accept(), bind(), connect(), recv with MSG_WAITALL)
can possibly return a socket.error with errno set to EAGAIN.
('resource temporarily unavailable').
If this happens you should use a select() on the socket to
wait until it's done with the requested operation.
--Irmen
p.s.
I've looked at twisted before posting this post. I've seen they
impelement alot of application level protocols but I didn't see much
treatment for low level raw network data, not to mention that it's a
way way over kill for what I'm asking to achieve. Twisted does have a
subproject called Twisted Pair: Low-level networking but sadly it's
unmaintained and undocumented.
Mike
Maxim.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Getting word frequencies from files which are in folder.
[EMAIL PROTECTED] wrote: Hello to all, I'm beginer in learning Python I wish somebody help me with solving this problem. I would like to read all text files wchich are in some folder. For this text files I need to make some word frequencies using defined words like buy, red, good. If some file don't have that word will get 0 for this frequency. It shoud be stored in array. If I have alredy got frequencies for every file in folder, my array wrote to text file. This sounds suspiciously like a homework assignment. I don't think you'll get much help for this one, unless you show some code you wrote yourself already with a specific question about problems you're having --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I find out the size of a file
[EMAIL PROTECTED] wrote: How can I find out the size of a file in a disk in python? os.path.getsize(filename) -Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Create new processes over telnet in XP
Shane Geiger wrote: This reminds me of something I once wanted to do: How can I install Python in a totally non-gui way on Windows (without the use of VNC)? I think I was telnetted into a computer (or something like that) and I was unable to run the usual Python installer because it uses a GUI. Python uses a MSI (microsoft installer) based installer on windows. This was introduced in version 2.5 I believe. For MSI installers there's the standard MSI-way to perform a silent install. Google for it, I don't know what the command line switch(es) are, but they're there. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: replace illegal xml characters
killkolor wrote: Does InDesign export broken XML documents? What exactly is your problem? yes, unfortunately it does. it uses all possible unicode characters, though not all are alowed in valid xml (see link in the first post). Are you sure about this? Could you post a small example? If this is true, don't forget to file a bug report with Adobe too. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
ANN: Pyro 3.6 (remote objects)
I'm happy to announce Pyro 3.6 -- Python's own powerful remote method invocation technology! You can get it via http://pyro.sourceforge.net, then go to the SF project homepage download area. It has many new features and improvements over Pyro 3.5, which was released more than a year ago. (Sorry about the glacial update rate) Most important new stuff: - python 2.5 compatibility fixes - multithreading stability fixes - mobile code race condition fixes - a few new examples - various stability improvements and other fixes - new looks for the manual The full list is far too long to include here. Please view it online: http://pyro.sourceforge.net/manual/12-changes.html#latest Have fun, and thanks for your interest, support, and feedback! --Irmen de Jong --- What is Pyro? Pyro is an acronym for PYthon Remote Objects. Pyro is an advanced and powerful Distributed Object Technology system written entirely in Python, that is designed to be very easy to use. It is extremely easy to implement a distributed system with Pyro, because all network communication code is abstracted and hidden from your application. You just get a remote Python object and invoke methods on the object on the other machine. Pyro offers you a Name Server, an Event Service, mobile objects, remote exceptions, dynamic proxies, remote attribute access, automatic reconnection, a very good and detailed manual, and many examples to get you started right away. -- http://mail.python.org/mailman/listinfo/python-list
Re: distributed queue?
Paul Rubin wrote: Does anyone have an implementation of a distributed queue? I.e. I have a long running computation f(x) and I'd like to be able to evaluate it (for different values of x) on a bunch of different computers simultaneously, the usual worker thread pattern except distributed across a network. I guess this is pretty easy to write with a centralized socket listener that dispatches requests through a Queue to multiple threads on the same machine, each talking synchronously to a server socket. I wonder if something like it already exists. I see a little bit of discussion in the newsgroup archive but no obvious pointers to code. Thanks. Pyro (http://pyro.sf.net) contains 2 examples that do just this. One is a distributed merge sort / md5 cracker, the other is distributed prime factorization of a set of numbers. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: How to test if a key in a dictionary exists?
Frank wrote:
Hi,
does anyone know how one can test if, e.g., a dictionary 'name' has a
key called 'name_key'?
name_key in name
e.g.
name={john: 42}
john in name
True
julie in name
False
--Irmen
--
http://mail.python.org/mailman/listinfo/python-list
Re: pickle problem - frexp() out of range
ahaldar wrote: Hi: I have some large data structure objects in memory, and when I attempt to pickle them, I get the following error: SystemError: frexp() out of range Are there some objects that are just too large to serialize, and if so, is there an easy workaround without breaking up the object and reconstructing it during deserialization? Here's the code I use to pickle the object: f = open(dir+file, w+b) pickle.dump(structure, f, protocol=2) # throws error f.close() - abhra Could it be that your data contains floating point numbers, where at least one of them is Inf or NaN? I think these floats cannot be pickled reliably. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Possible to set cpython heap size?
Andy Watson wrote: Why do you want that? And no, it is not possible. And to be honest: I have no idea why e.g. the JVM allows for this. Diez The reason why is simply that I know roughly how much memory I'm going to need, and cpython seems to be taking a fair amount of time ^ extending its heap as I read in content incrementally. First make sure this is really the case. It may be that you are just using an inefficient algorithm. In my experience allocating extra heap memory is hardly ever noticeable. Unless your system is out of physical RAM and has to swap. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Forking SocketServer daemon -- updating state
Reid Priedhorsky wrote: Another possibility is that the signal handler simply sets a needs_update flag, which I could check for in a handle_request() loop. The disadvantage here is that the update wouldn't happen until after the next request is handled, and I would like the state to be available for that next request. A solution might be to send a signal followed by a dummy request, which seems a bit awkward. Any other ideas or suggestions? Just send a special type of request that signifies that an update is wanted, and act on that? Basically you need to find a way to let two processes talk to each other. There are a lot of possibilities here (IPC). The simplest would be to reuse the one you already have (the request handler!). Another solution might be to use Pyro. Or just open a custom socket yourself to send messages. Or stick with your idea of using a signal handler. But I don't know if you can let a signal handler run for a long time (as you already asked yourself), and it won't be portable to Windows for instance. You could maybe use threads instead, and then use some form of thread synchronization primitives such as threading.Event (threads would remove the need to do IPC). Hope this helps a bit, --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: message processing/threads
Jonathan Curran wrote: I need a program running in the background to process messages (FIFO order) which I would send using soap/xmlrpc/pyro (haven't decided yet). According to my thinking I would need to make this a threaded application. One thread to process the messages and the other thread(s) would be used to listen for messages and insert it into the message queue. Is my thinking correct? Is there a better way to do such a thing? For your interest: Pyro includes a server by itself that takes care of all of this. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
ANN: Pyro 3.6 beta
It is with great pleasure that I announce that the release of a beta version of Pyro 3.6-- the greatest Pyro release yet! What is Pyro? Pyro is short for PYthon Remote Objects. It is an advanced and powerful Distributed Object Technology system written entirely in Python, that is designed to be very easy to use. Never worry about writing network communication code again. Go to Pyro's Sourceforge download page to get it, and please give it a good testing. It would be great to hear of any issues or problems you encounter, before putting out the final 3.6 release. Pyro has been a Sourceforge project for over 6 years now. http://pyro.sourceforge.net http://sourceforge.net/projects/pyro I have made a HUGE amount of improvements and fixes since the previous version, 3.5, which was released about 14 months ago (sorry!). If you want to review what has been done, read the 'changes' chapter in the manual, and/or visit Pyro's todo wiki page: http://www.razorvine.net/python/PyroTodoList Thanks for your support, and I'm looking forward to release a final Pyro-3.6 version soon ! Sincerely, --Irmen de Jong PS: Sourceforge's shell access is down at the moment so I can't update the Pyro web page itself yet. -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
ANN: Pyro 3.6 beta released
It is with great pleasure that I announce that the release of a beta version of Pyro 3.6-- the greatest Pyro release yet! What is Pyro? Pyro is short for PYthon Remote Objects. It is an advanced and powerful Distributed Object Technology system written entirely in Python, that is designed to be very easy to use. Never worry about writing network communication code again. Go to Pyro's Sourceforge download page to get it, and please give it a good testing. It would be great to hear of any issues or problems you encounter, before putting out the final 3.6 release. Pyro has been a Sourceforge project for over 6 years now. http://pyro.sourceforge.net http://sourceforge.net/projects/pyro I have made a HUGE amount of improvements and fixes since the previous version, 3.5, which was released about 14 months ago (sorry!). If you want to review what has been done, read the 'changes' chapter in the manual, and/or visit Pyro's todo wiki page: http://www.razorvine.net/python/PyroTodoList Thanks for your support, and I'm looking forward to release a final Pyro-3.6 version soon ! Sincerely, --Irmen de Jong PS: Sourceforge's shell access is down at the moment so I can't update the Pyro web page itself yet. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to Speed Up Internet Searches??
Patrick Klos wrote: In article [EMAIL PROTECTED], [EMAIL PROTECTED] wrote: How to Speed Up Internet Searches?? When you go to a web site, the first thing that happens is that. and for networking tips see at : : : Please don't post this kind of stuff here any more. It's off topic and unappreciated! Not to mention so ugly and extremely outdated that I had to read the page twice to see what it was about. It actually contains Hayes modem commands. (sorry, I went there out of curiousity...) -Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Python, readline and OS X
Ron Garret wrote: I have installed Python 2.5 on my new Intel Mac but I can't for the life of me get readline to work. I have libreadline installed, I've tried copying readline.so from my Python 2.3 installation into 2.5, I've searched the web, and no joy. Could someone please give me a clue? rg Does the info in a blog article that I wrote help? http://www.razorvine.net/frog/user/irmen/article/2006-05-08/87 I used this when I compiled my Python 2.5 on my mac, and it seemed to work ;-) I'm now using the python.org binary distribution though and that seems to contain a working readline as well ? --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Calculating future dates
Toine wrote: Hello, I'm new to Python so please bare with me... I need to calculate a date that is exactly 31 days from the current date in -MM-DD format. I know that date.today() returns the current date, but how can I add 31 days to this result? I'm sure this task is simple, but I haven't been able to figure it out. import datetime print datetime.date.today()+datetime.timedelta(days=31) 2007-03-05 ---Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: socket.inet_ntop, and pton question
Andrew wrote: Hi Are these functions (inet_ntop(), inet_pton()) from the socket library supported on Windows. If not is there an equivalent for them using Windows Ive seen mention of people creating their own in order to use them Appreciate the help ty Why didn't you just try: [E:\Projects]python Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import socket socket.inet_aton(127.0.0.1) '\x7f\x00\x00\x01' socket.inet_ntoa(_) '127.0.0.1' This is on windows xp. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: socket.inet_ntop, and pton question
Gabriel Genellina wrote: But these are not the requested functions, inet_ntop() and inet_pton(): py socket.inet_ntop Traceback (most recent call last): File stdin, line 1, in ? AttributeError: 'module' object has no attribute 'inet_ntop' Oops, my bad. Should have had more coffee before replying I guess. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: newb: Can I use PYRO
johnny wrote: What I want to do is the following: Web user uploads a word doc (web app written in php), and I need it to move the uploaded word doc, on to another machine and conver it to pdf. Then update the database and allow immediate pdf download. I am thinking of using ftp from machine 1 - machine 2, then convert doc to pdf on machine 2, using process or thread. What is the best way to go about doing this, process or threads or pyro? Pdf is created by calling commands on the command line, through python script. Even though you are using Python on the machine that converts things to PDF, you're stuck with PHP on the web server. You cannot use Pyro with PHP. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: client/server design and advice
John Henry wrote: On the subject of passing things around, is there a no brainer way of sending files back and forth over Pyro? I am currently using a shared drive to do that. May be I missed that feature? Sending files around is just a special case of passing large amounts of data to other systems. There is no built in support for file transfer in Pyro. It's rather easy to make a file transfer tool though, there is an example in the Pyro distribution that shows how this *may* be done. But choose the right tool for the job, things like rsync or even ftp may be preferable for just file transfer! If you use a shared drive, you could use Pyro to 'tell' the receiving side that the file is there with this-and-this name, to trigger a download. When the receiver is done copying the file, it can signal back and the sender can delete the file. But that's just an idea. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: client/server design and advice
TonyM wrote: Lastly, as far as the networking goes, i have seen posts and such about something called Pyro (http://pyro.sourceforge.net) and wondered if that was worth looking into for the client/server interaction. I'm currently busy with a new version of Pyro (3.6) and it already includes a new 'distributed computing' example, where there is a single dispatcher service and one or more 'worker' clients. The clients request work 'packets' from the dispatcher and process them in parallel. Maybe this is a good starting point of your system? Current code is available from Pyro's CVS repository. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: client/server design and advice
bruce wrote: hi irmen... happened to come across this post. haven't looked at pyro. regarding your 'work packets' could these essentially be 'programs/apps' that that are requested by the client apps, and are then granted by the dispatch/server app? Pyro supports a limited form of mobile code i.e. the automatic transfering of Python modules to the other side. But it has a few important limitations and there's the security aspect as well. It's really better if you can stick to passing data around, not code... ;-) i'm considering condor (univ of wisconsin) but am curious as to if pyro might also work. Not familiar with this. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic function execution
Andy Wu wrote:
Say I have a string 'minutes' and a integer 30, now I need to call the
func this way: func(minutes = 30), how do I do this?
d={minutes: 30}
func(**d)
This is extended call syntax. You can read more about this when
you look up the (deprecated) apply function in the manual.
--Irmen
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why does this code crash python?
[EMAIL PROTECTED] wrote: I am trying to make a program that will basically simulate a chess clock in python. To do this I have two threads running, one that updates the currently running clock, and one that watches for a keypress. I am using the effbot Console module, and that is where I get the events for the keypresses. But when I press space it crashes shortly after. The program stops, the Console closes, and windows says that the program pythonw.exe has had an unexpected reason to shut down. I am using python2.5. Try not (re)using an object created in a certain thread in another thread. You're creating a console object c and use it in both threads. Try creating the console object for each thread by itself, i.e. remove the global c = Console.getconsole() and replace it by a getconsole() inside each thread class. I don't have the Console module so I don't know if it fixes things for you, but give it a try :) --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: profanity on comp.lang.python (was Re: Pyro stability)
Jorge Godoy wrote: Cliff Wells [EMAIL PROTECTED] writes: I think this sums up my point of view as well (although I would have used around 3215 more words to say it). H... Putting this on the discussion of the week: you'd have used range(3215) or xrange(3215) more words? ;-) Actually I've just replaced a loop in Pyro from using range to using xrange because of memory issue when you wanted to run it nearly indefinitely. So there. Finally back on the original subject ;-) Thanks for the praise about Pyro but yeah, I would have been equally happy if it was put without using the f word :) Cheers --Irmen. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pyro stability
writeson wrote: Irmen, Thanks, you're very good about answering Pyro related questions! Well, I do have an advantage here, being Pyro's author... :) --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Pyro stability
writeson wrote: [some questions about Pyro] I've replied to this on Pyro's mailing list. -Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: How set the source IP adress
Maksim Kasimov wrote:
Hi,
how to set source ip-address when do __socket.connect((host, port))
on a machine that have a several ip-adresses?
many thanks for advice.
__socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
__socket.connect((host, port))
sock.connect ( ('11.22.33.44', ) )
i.e. just put the ip address as string parameter into the connect address tuple
(where you wrote: host)
--Irmen
--
http://mail.python.org/mailman/listinfo/python-list
Re: Stylesheet not respected
Suren wrote: I was able to the see weird stylesheet behavior on opera, IE and mozilla under mod_python. [snip] I'm 99% sure this has nothing to do with Python but is just an error in your CSS file. Show the CSS so we might be able to see the problem. One thing to check for though is that your web server is serving CSS files with the correct content type (text/css and not text/plain or somthing equally silly). --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Socket module bug on OpenVMS
Fredrik Lundh wrote: Irmen de Jong wrote: This also raises the question to what extent Python itself should work around platform specific peculiarities, such as this one. There's another problem with socket code on Windows and VMS systems, where you get strange exceptions when using a too big recv() buffer. what's a strange exception and a too big buffer? The exceptions are MemoryError (I know this one for sure) and a socket.error I believe (can't remember exactly, and I don't have a VMS machine to try to reproduce). Too big buffer means anything above 64 kilobyte or so. You can find a lot of reports about this happening on Windows at least. From user reports I've learned that VMS also has similar problems with recv buffer sizes above a certain size. Things like this force me into writing all sorts of error checking code or platform specific functions, to work around these bugs. Just for what was supposed to be a simple socket recv() and a simple socket send()... if you want buffering, use makefile(). relying on platform-specific behaviour isn't a good way to write portable code. I'm not sure if makefile() would shield me from the problems I experienced (I could try I suppose) but your second remark is exactly my point! I don't *want* to code around platform-specific behavior. I *want* my code to be portable. And I expected it to be portable by just using a regular recv() call... However as it is now, Python's socket module exposes platform specific (and troublesome) behavior, so I have to write various workarounds to make my code run without errors on multiple platforms... Regards, --Irmen de Jong -- http://mail.python.org/mailman/listinfo/python-list
Re: Socket module bug on OpenVMS
Martin v. Löwis wrote: Irmen de Jong schrieb: In my opinion Python's socket module itself could implement these workarounds. That would make user code a lot cleaner and less error prone, and more portable. What do other people think? It depends: when VMS defines MSG_WAITALL, but doesn't implement it correctly, then Python probably shouldn't expose it. That would be one way of solving at least my troubles, because I already check for the availability of MSG_WAITALL and revert to a custom recv() loop otherwise. As others have already reported, at this time Python (well, the build process) merely checks for the availability of the MSG_WAITALL symbol in the socket.h C-header file Implementing recv() splitting is *not* something that Python could do itself. The question is how you do error reporting for partial results [..snip..] Mm, tricky. Hadn't thought of that. Although I'm not yet convinced about the all-or-nothing failure model you were talking about. How can I know if recv() fails with a MemoryError (or socket error) that it actually didn't receive anything? Is that even an assumption I can rely on? If it isn't, splitting up the recv() internally (into smaller blocks) wouldn't be any different, right? Perhaps you had some different work-around in mind? Nope, I was indeed thinking about splitting up the recv() into smaller blocks internally. Maybe a more elaborate check in Python's build system (to remove the MSG_WAITALL on VMS) would qualify as some sort of work-around as well. Although that would not yet solve the errors you get when using too big recv() buffer sizes. Cheers, --Irmen. -- http://mail.python.org/mailman/listinfo/python-list
Socket module bug on OpenVMS
Hi, Recently I was bitten by an apparent bug in the BSD socket layer on Open VMS. Specifically, it appears that VMS defines MSG_WAITALL in socket.h but does not implement it (it is not in the documentation). And I use the socket.MSG_WAITALL flag on my recv() calls... and then they crash on OpenVMS. I don't have access to an OpenVMS machine myself so could someone else that has (or has more knowledge of it) shed some light on it? This also raises the question to what extent Python itself should work around platform specific peculiarities, such as this one. There's another problem with socket code on Windows and VMS systems, where you get strange exceptions when using a too big recv() buffer. Things like this force me into writing all sorts of error checking code or platform specific functions, to work around these bugs. Just for what was supposed to be a simple socket recv() and a simple socket send()... In my opinion Python's socket module itself could implement these workarounds. That would make user code a lot cleaner and less error prone, and more portable. What do other people think? Regards --Irmen de Jong -- http://mail.python.org/mailman/listinfo/python-list
Altering the way exceptions print themselves
Hello, I want to alter the way exceptions print themselves. More specifically, I'd like to extend the __str__ method of the exception's class so that it is printed in a different way. I used to replace the __str__ method on the exception object's class by a custom method, but that breaks my code on Python 2.5 because of the following error I'm getting: TypeError: can't set attributes of built-in/extension type 'exceptions.TypeError' Well, ok. So I tried to improve my code and not alter the class, but only the exception object. I came up with this: def __excStr__(self): return [[[EXCEPTION! +self.originalStr()+]] ... some code that raises an exception 'ex' import new newStr = new.instancemethod( __excStr__, ex, ex.__class__) ex.__str__=newStr On Python 2.4 the above code works as I expect, however on Python 2.5 it doesn't seem to have any effect... What am I doing wrong? Or is there perhaps a different way to do what I want? Thanks! --Irmen de Jong -- http://mail.python.org/mailman/listinfo/python-list
Re: Socket module bug on OpenVMS
Jean-Paul Calderone wrote: I think everyone can agree that Python shouldn't crash. Well, it doesn't really crash in a bad way, in my example: it doesn't work because it simply raises a socket exception all the time. Whether Python should propagate other kinds of errors from the underlying platform is a harder question though. At the lowest level interface, it seems to me that it _should_ propagate them. If there is a general way to handle them, then a higher layer can be built on top of that lowest level which does so. We already have a low-level _socket module (builtin) and a higher level socket module (implemented in python in the std lib)... I could imagine that the socket module would offer methods that can deal with platform bugs such as the two I mentioned in my original posting. I have never used _socket directly by the way (and don't think anyone else ever did?) --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Socket module bug on OpenVMS
Jean-François Piéronne wrote: Which Python version, OpenVMS version, IP stack and stack version? OpenVMS 7.3-2, Python 2.3.5, no idea about IP stack version. If you think this is a Python on OpenVMS problem, send me a small reproduced anf I will take a look. I don't have any small case lying around (I can't reproduce it myself because I lack access to an openVMS machine) but I think you should be able to reproduce it by slightly altering one of the socket examples that comes with Python. Just add the MSG_WAITALL to the recv() call: something = sock.recv(somesize, socket.MSG_WAITALL) and you should see it crash with a socket exception. Mail me offline if you still need running example code (that I think would expose the problem). If you think it's a OpenVMS problem and if you can provide a simple reproducer and have a support contract I suggest you call HP, but I suspect that if it's not documented the reply would be not (yet?) supported. I don't have anything to do with HP... the person that reported the problem to me has, however. He's already aware of the problem. May be a workaround is to not use MSG_WAITALL (currently) on OpenVMS and in next version I will not defined MSG_WAITALL in the socket module on OpenVMS. How can I detect that I'm running on OpenVMS? --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Socket module bug on OpenVMS
Jean-Paul Calderone wrote: On Sun, 22 Oct 2006 19:58:44 +0200, Irmen de Jong [EMAIL PROTECTED] wrote: Jean-Paul Calderone wrote: I think everyone can agree that Python shouldn't crash. Well, it doesn't really crash in a bad way, in my example: it doesn't work because it simply raises a socket exception all the time. Okay. I assumed you meant the interpreter died with a SIGSEGV or something similar. If the call simply raises an exception, then I'm not sure I see a real problem here. Hmm.. it is not supposed to raise the exception (the code is valid but just breaks on windows or VMS). On the other hand, the underlying platform goes bad and what else could Python do? (at this point) What'd be really nice is a direct wrapper around recv(2), send(2), etc, with a simple socket-object layer on top of that, with timeouts or error handling layered on top of that. I agree, that would be nice indeed. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Sending binary pickled data through TCP
David Hirschfield wrote: I have a pair of programs which trade python data back and forth by pickling up lists of objects on one side (using pickle.HIGHEST_PROTOCOL), and sending that data over a TCP socket connection to the receiver, who unpickles the data and uses it. So far this has been working fine, but I now need a way of separating multiple chunks of pickled binary data in the stream being sent back and forth. [...] Save yourself the trouble of implementing some sort of IPC mechanism over sockets, and give Pyro a swing: http://pyro.sourceforge.net In Pyro almost all of the nastyness that is usually associated with socket programming is shielded from you and you'll get much more as well (a complete pythonic IPC library). It may be a bit heavy for what you are trying to do but it may be the right choice to avoid troubles later when your requirements get more complex and/or you discover problems with your networking code. Hth, ---Irmen de Jong -- http://mail.python.org/mailman/listinfo/python-list
Re: How to ask sax for the file encoding
Edward K. Ream wrote: What suits me best is what the *user* specified, and that got put in the first xml line. I'm going to have to parse this line myself. Please consider adding some elements to the document itself that describe the desired output format, such as: ... output encodingutf-8/encoding /output ... This allows the client to specify the encoding it wants to receive the document in, even if it's different than the encoding it used to make the first document. More flexibility. Less fooling around. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there an alternative to os.walk?
Bruce wrote: Hi all, I have a question about traversing file systems, and could use some help. Because of directories with many files in them, os.walk appears to be rather slow. Provide more info/code. I suspect it is not os.walk itself that is slow, but rather the code that processes its result... I`m thinking there is a potential for speed-up since I don`t need os.walk to report filenames of all the files in every directory it visits. Is there some clever way to use os.walk or another tool that would provide functionality like os.walk except for the listing of the filenames? You may want to take a look at os.path.walk then. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: How to ask sax for the file encoding
Edward K. Ream wrote: Please consider adding some elements to the document itself that describe the desired output format, Well, that's what the encoding field in the xml line was supposed to do. As others have tried to explain, the encoding in the xml header is not part of the document data itself, it says something about the data. It would be a bad design decision imo to rely on this meta information if you really meant that information to be part of the data document. Not a bad idea though, except it changes the file format, and I would really rather not do that. XML allows you to easily skip any elements that you think you don't need. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: php and python: how to unpickle using PHP?
Ted Zeng wrote: Hi, I store some test results into a database after I use python To pickle them (say, misfiles=['file1','file2']) Now I want to display the result on a web page which uses PHP. How could the web page unpickle the results and display them? Is there a PHP routine that can do unpickle ? Ted zeng Pickle is Python specific. You may be able to write something that can unpickle the basic datatypes but I wouldn't bother with it. Try to find a language neutral marshaling format (yaml? xml? whatever) and access that from both Python and PHP... --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: creating a small test server on my local computer
John Salerno wrote: Ok, this is completely unnecessary so I don't intend to get into stuff that's beyond my skill, but I'm wondering how simple it would be to use Python to create a server that runs on my computer so I can test my webpages (because otherwise I have to keep sending them to our IT person so he can upload them to the server). Why do you need to use Python for the server? The catch is that I need a server that supports SSI and I have no clue how to write something like this. If it's already been included in the standard library, then that would be great, otherwise I won't really mess with it. Apache supports server side includes. And is fairly straightforward to install (or even already installed on most linux systems that I know of) Configuring can be a bit troublesome depending on your demands, but there is lots and lots of information available. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: new string method in 2.5 (partition)
Terry Reedy wrote: Bruno Desthuilliers [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Err... is it me being dumb, or is it a perfect use case for str.split ? s.partition() was invented and its design settled on as a result of looking at some awkward constructions in the standard library and other actual use cases. Sometimes it replaces s.find or s.index instead of s.split. In some cases, it is meant to be used within a loop. I was not involved and so would refer you to the pydev discussions. While there is the functional aspect of the new partition method, I was wondering about the following /technical/ aspect: Because the result of partition is a non mutable tuple type containing three substrings of the original string, is it perhaps also the case that partition works without allocating extra memory for 3 new string objects and copying the substrings into them? I can imagine that the tuple type returned by partition is actually a special object that contains a few internal pointers into the original string to point at the locations of each substring. Although a quick type check of the result object revealed that it was just a regular tuple type, so I don't think the above is true... --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: new string method in 2.5 (partition)
Gabriel Genellina wrote: Nope, a python string has both a length *and* a null terminator (for ease of interfacing C routines, I guess) so you can't just share a substring. Ofcourse, that makes perfect sense. Should have thought a little bit further myself :) --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Python blogging software
Cliff Wells wrote: I'm currently using Frog, and it's decent, but lacks some fundamental features (tags for one). Since Irmen is probably going to scrap it anyway, I'm kind of fishing about for something new. That is not really true. I won't scrap Frog. One of the reasons would be that I'm using it myself ;-) Perhaps you confused it with the possible scrapping of the Snakelets appserver it runs on? I'm thinking about rebuilding Frog on one of the more established servers such as Turbogears. But haven't had the time to start this. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: about daemons and IPC
[EMAIL PROTECTED] wrote: Hey people! For the first time I'm doing a client/server application, and I'm really confused with IPC stuff. [...] Any suggestions? http://pyro.sourceforge.net depending on your needs --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Python linker
[EMAIL PROTECTED] wrote: I love python - I use it as a utility language to complement my C# programming every day. However, the reason I do not use it as my primary language is - surprise, surprise - not its lack of static type checking, but the size of standalone executes (which embed the python runtime). Why not just install Python itself, so that all of your python programs are only a few Kb each. Because they all use the single shared installation. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Headers for Form Submision, and also HTTPrequests
[EMAIL PROTECTED] wrote: form action=http://login.myspace.com/index.cfm?fuseaction=login.process; method=post name=theForm id=theForm input type=text name=email input type=password name=password input type=submit value=Login/ What happens when you add the form param submit with value Login to the request you're doing? Perhaps the page needs this third parameter --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: mirroring object attributes using xml-rpc
[EMAIL PROTECTED] wrote: but why can't I get the value of i like this? c.i How can I implement such behaviour? Not supported by XMLRpc. Switch to Pyro: http://pyro.sourceforge.net --Irmen -- http://mail.python.org/mailman/listinfo/python-list
socket close problems with Python 2.5b1
I'm having some troubles with closing sockets using Python 2.5b1 Simply closing a client socket (on the server side) doesn't seem to actually shutdown the socket anymore. A connected client simply hangs, while with older Pythons it aborted with a socket close error. Can someone confirm this for me? Or shed some light on it? More info and a test program is available in the following but report: http://sourceforge.net/tracker/index.php?func=detailaid=1513223group_id=5470atid=105470 Thanks, --Irmen. -- http://mail.python.org/mailman/listinfo/python-list
Re: locating strings approximately
BBands wrote: I'd like to see if a string exists, even approximately, in another. For example if black exists in blakbird or if beatles exists in beatlemania. The application is to look though a long list of songs and return any approximate matches along with a confidence factor. I have looked at edit distance, but that isn't a good choice for finding a short string in a longer one. I have also explored difflib.SequenceMatcher and .get_close_matches, but what I'd really like is something like: a = FindApprox(beatles, beatlemania) print a 0.857 Any ideas? jab I collected a few pointers in this article: http://www.razorvine.net/frog/user/irmen/article/2005-05-28/53 It contains one or two additional methods besides the ones you mentioned. Sorry, it's in Dutch, but you can find the links without much trouble i guess. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Network Programming in Python
[EMAIL PROTECTED] wrote: Really, was that so hard? Python makes sockets a total breeze. You can write an 80's style HTTP server in less than a page of code. But making a *good* 80's style http/socket server is a lot of work. Better pick one of the high level protocols built on top of it, to shield you from the gory details of raw socket programming. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: socket programming question
Kiran wrote: Hello All, My question is, is it possible to make python do some other processing while it is waiting for a socket to timeout? sure, you have to use threads and/or use asynchronous socket programming. Google is your friend. --Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: billionlaughs, on me :-(( - on winXP help!
[EMAIL PROTECTED] wrote: Fredrik Lundh wrote: [EMAIL PROTECTED] wrote: Looking up different blogs etc on xml parsers I wind up on the site of Computer stuff, www.razorvine.net. And in the browser torture section, the billionlaughs.xml link (that you shouldn't use if you don't know what your doing, as was my case (mea culpa)) ... reboot as usual, and the next time you see a link that says do NOT click if you don't know what you're doing, don't click on it. /F Right ;-) and thanks for your very fast reply ! The site is mine. And sorry for crashing your computer... But there really is a warning there to not click on it, like Fredrik pointed out ;-) You *can* try the millionlaughs version though, that more or less shows what the problem is. And it is small enough to not crash your browser. Right click on any of the link and save link as to download the XML file for viewing in your favorite editor :) Cheers, --Irmen de Jong -- http://mail.python.org/mailman/listinfo/python-list
