New parallel computing module for Python
The latest development release of ScientificPython (2.7), which is available now at http://sourcesup.cru.fr/projects/scientific-py/ contains a new module for parallel computing using a master-slave model: a master process defines computational tasks, and any number of slave processes execute them. The main advantages of this approach are simplicity and the possibility to run a variable number of slave processes. Slave processes can even be killed during execution; as long as at least one slave process remains, the computation will terminate correctly. Here is a very simple example that illustrates parallel computation using the new module: from Scientific.DistributedComputing.MasterSlave \ import MasterProcess, SlaveProcess from Scientific import N import sys class Master(MasterProcess): def run(self): for i in range(5): # request a computation task of type sqrt task_id = self.requestTask(sqrt, float(i)) for i in range(5): # retrieve the result from a sqrt computation task task_id, tag, result = self.retrieveResult(sqrt) print result class SquareRoot(SlaveProcess): # implement the computation task sqrt def do_sqrt(self, x): return (x, N.sqrt(x)) if len(sys.argv) == 2 and sys.argv[1] == master: master = True elif len(sys.argv) == 2 and sys.argv[1] == slave: master = False else: print Argument must be 'master' or 'slave' raise SystemExit if master: process = Master(demo) else: process = SquareRoot(demo) process.start() The communication between processes is handled by Pyro (pyro.sourceforge.net), which has to be installed as well. Comments are welcome! -- - Konrad Hinsen Centre de Biophysique Moléculaire, CNRS Orléans Synchrotron Soleil - Division Expériences Saint Aubin - BP 48 91192 Gif sur Yvette Cedex, France Tel. +33-1 69 35 97 15 E-Mail: [EMAIL PROTECTED] - -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
ANN: amplee 0.3.3
Hi all, I'm happy to introduce the release of Amplee 0.3.3, a Python implementation of the Atom Publishing Protocol. This release is a bug fixes but with a few new features as well: * Improved support of the Amazon S3 service as a storage * Support for the Hachoir library which should offer more format supported as members * Fixed the AtomMember class * Improved the potential of the different callbacks on creation, update and deletion of resources. * Added lots of docstrings (still not enough though) * Fixed quite a few bunch of bugs (thanks to David Turner) * Fix the WSGI handler to use the new routing_args spec. * Largely improved the demo as well This release is getting closer to what the final API will look like and I think it's a pretty stable version. == Download == * easy_install -U amplee * Tarballs http://www.defuze.org/oss/amplee/ * svn co https://svn.defuze.org/oss/amplee/ == Documentation == http://trac.defuze.org/wiki/amplee http://www.defuze.org/oss/amplee/doc/html/ == TODO == * Add tests * Remove genser dependency * Improve documentation Have fun, -- Sylvain Hellegouarch http://www.defuze.org -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
Re: Yield
John Machin wrote: I would like to thanks Fredrik for his contribution to improve that. Call me crazy, but after an admittedly quick read, the version on the wiki seems to be about word for word with on the docs.python.org version. maybe he was thinking about the article I posted, or the FAQ link I posted in a followup: http://effbot.org/pyfaq/what-is-a-generator.htm which was based on my article and the glossary entry from the tutorial: http://effbot.org/pytut/glossary.htm#generator Could you please tell us how you think the wiki version is an improvement? an add comment link? direct URL:s for all concepts in the language? extensive hyperlinking? semantic retargetable markup? /F -- http://mail.python.org/mailman/listinfo/python-list
How to make a python file to a DLL?
Any solution? Thanks. -- http://mail.python.org/mailman/listinfo/python-list
matplotlib real time grap.
Hello at all.
I need to do a real time plot where on the frame i have this like limit
line:
import math
dati = []
for freq in range(int(freqiniziale), (int(freqfinale )+ 1)):
forza = float(massa) * ((2*math.pi*freq)**2)/10
dati.append(forza)
ax1 = subplot(111)
plot(dati)
xlabel('Frequenza HZ')
ylabel('Deg')
title('Grafico Calibrazione')
ax2 = twinx()
plot([0.1,0.2,0.3],'r-')
ylabel('Forza')
ax2.yaxis.tick_right()
#dlg = wx.FileDialog(self, Choose a file, ., , *.*,
wx.SAVE)
#try:
# if dlg.ShowModal() == wx.ID_OK:
#filename = dlg.GetPath()
# savefig(filename+'.pdf')
#finally:
# dlg.Destroy()
show()
with this i have the grap on the screen but i have to update every time
that i receive others data coming from two instruments (10 seconds),
how i can update the grap?
Regards
Luca
--
http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
timmy wrote: congraulations you have discovered loops and their misuse if you don't know what the phrase denial of service attack means, you can always google for it. /F -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml/ElementTree and .tail
Stefan Behnel wrote: If you want to copy part of of removed element back into the tree, feel free to do so. and that can of course be done with a short helper function. when removing elements from trees, I often set the tag for those elements to some garbage value during processing, and then call something like http://effbot.org/zone/element-bits-and-pieces.htm#cleanup to clean things up before serializing the tree. /F -- http://mail.python.org/mailman/listinfo/python-list
Re: Yield
On 16/11/2006 7:00 PM, Fredrik Lundh wrote: John Machin wrote: I would like to thanks Fredrik for his contribution to improve that. Call me crazy, but after an admittedly quick read, the version on the wiki seems to be about word for word with on the docs.python.org version. maybe he was thinking about the article I posted, or the FAQ link I posted in a followup: http://effbot.org/pyfaq/what-is-a-generator.htm which was based on my article and the glossary entry from the tutorial: http://effbot.org/pytut/glossary.htm#generator Quite possibly. Could you please tell us how you think the wiki version is an improvement? an add comment link? direct URL:s for all concepts in the language? extensive hyperlinking? semantic retargetable markup? Yes, they're great. Maybe I was thinking about the text contents only :-) Cheers, John -- http://mail.python.org/mailman/listinfo/python-list
Re: How to make a python file to a DLL?
many_years_after wrote: Any solution? python modules are usually shipped in ZIP archives, not DLLs. if you really need a DLL, you can use freeze and some custom hacking. or bytecode resources, and some more custom hacking. /F -- http://mail.python.org/mailman/listinfo/python-list
Re: More puzzling behavior while subclassing datetime
[EMAIL PROTECTED] wrote:
With assistance from Gabriel and Frederik (and a few old threads in
c.l.p.) I've been making headway on my specialized datetime class. Now
I'm puzzled by behavior I didn't expect while attempting to use some of
the alternate datetime constructors. Specifically, it appears if I
call GeneralizedTime.now() it calls the __new__ method of my class but
treats keyword arguments as if they were positional.
My class:
class GeneralizedTime(datetime):
def __new__(cls, time=None, *args, **kwargs):
datetime.datetime() takes these arguments: year, month, day[, hour[,
minute[, second[, microsecond[, tzinfo]), see
http://docs.python.org/lib/datetime-datetime.html
print time, args, kwargs
if isinstance(time, str):
timeValues, tzOffset = cls.stringToTimeTuple(time)
return datetime.__new__(cls, tzinfo=GenericTZ(tzOffset),
**timeValues)
elif isinstance(time, datetime):
timeValues = time.timetuple()[:6]
time.timetuple() does not exist, see
http://docs.python.org/lib/module-time.html, time is represented as a
tuple. checkout time.mktime() on how to convert to a tuple to a time
tzOffset = time.utcoffset()
return datetime.__new__(cls, tzinfo=GenericTZ(tzOffset),
*timeValues)
elif time is None:
print Still gotta figure out now to do this one...
else:
raise Invalidtime(time)
@staticmethod
def stringToTimeTuple(timeString):
... regex that parses timeString ...
GeneralizedTime.today()
2006 (11, 16, 0, 35, 18, 747275, None) {}
Traceback (most recent call last):
File stdin, line 1, in ?
File gentime.py, line 106, in __new__
raise InvalidTime(time)
gentime.InvalidTime: 2006
So it appears the time tuple is being passed to
GeneralizedTime.__new__, but the first value is being assigned to the
time argument.
Is this a side effect of how datetime is implemented? Or am I doing
something screwy?
Thanks!
-Ben
A very cutback part of your code gets the basics working:
from datetime import datetime
class Invalidtime(Exception):
pass
class GeneralizedTime(datetime):
def __new__(cls, *args, **kwargs):
if isinstance(args, tuple):
return datetime.__new__(cls, *args)
else:
raise Invalidtime(args)
t = GeneralizedTime.today()
print t.year
print t.month
print t.day
print t.hour
print t.minute
print t.second
print t.microsecond
print t.tzinfo
--
http://mail.python.org/mailman/listinfo/python-list
Re: More puzzling behavior while subclassing datetime
[EMAIL PROTECTED] wrote:
With assistance from Gabriel and Frederik (and a few old threads in
c.l.p.) I've been making headway on my specialized datetime class. Now
I'm puzzled by behavior I didn't expect while attempting to use some of
the alternate datetime constructors. Specifically, it appears if I
call GeneralizedTime.now() it calls the __new__ method of my class but
treats keyword arguments as if they were positional.
My class:
class GeneralizedTime(datetime):
def __new__(cls, time=None, *args, **kwargs):
print time, args, kwargs
if isinstance(time, str):
timeValues, tzOffset = cls.stringToTimeTuple(time)
return datetime.__new__(cls, tzinfo=GenericTZ(tzOffset),
**timeValues)
elif isinstance(time, datetime):
timeValues = time.timetuple()[:6]
tzOffset = time.utcoffset()
return datetime.__new__(cls, tzinfo=GenericTZ(tzOffset),
*timeValues)
elif time is None:
print Still gotta figure out now to do this one...
else:
raise Invalidtime(time)
@staticmethod
def stringToTimeTuple(timeString):
... regex that parses timeString ...
GeneralizedTime.today()
2006 (11, 16, 0, 35, 18, 747275, None) {}
Traceback (most recent call last):
File stdin, line 1, in ?
File gentime.py, line 106, in __new__
raise InvalidTime(time)
gentime.InvalidTime: 2006
So it appears the time tuple is being passed to
GeneralizedTime.__new__, but the first value is being assigned to the
time argument.
Is this a side effect of how datetime is implemented?
Yes. Consider:
def today(time=None, *args):
... print time = , time, args = , args
...
today(2006, 11, 16)
time = 2006 args = (11, 16)
To fix the issue you'll probably have to remove the time=None parameter from
GeneralizedTime.__new__() and instead extract it from args or kwargs.
Peter
--
http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
On Thu, 16 Nov 2006 17:44:37 +1000, timmy wrote: congraulations you have discovered loops and their misuse Did you have a point in your utterly inane comment, or did you just want to see your name on Usenet? In any case, it isn't just loops that are dangerous. print 2**512**512 No loop there, but it will operate as a lovely DoS attack if you run it. The Original Poster is suggesting running UNTRUSTED code. That means you have to assume that it will be actively hostile, but even if it isn't deliberately hostile, there will be bugs which the developer can't control. He wants to run this untrusted (hostile or buggy or both) code in an environment where it can't do bad things. Bad things include Denial of Service attacks. So, Timmy, let's hear your brilliant scheme for preventing DoS attacks when running hostile code in Python. -- Steven D'Aprano -- http://mail.python.org/mailman/listinfo/python-list
Re: Will GPL Java eat into Python marketshare?
Dennis Lee Bieber wrote: On Wed, 15 Nov 2006 22:41:19 GMT, Maurice LING [EMAIL PROTECTED] declaimed the following in comp.lang.python: I'm hoping for a more optimistic outcome that this may open a possibility for tigher interoperability between java programs and python programs. That is, run java class files or java codes natively on python VM. Is this still a blue sky dream? Most unlikely to happen... I don't really see anyone going to the effort to change the javac back-end to target a totally different runtime engine. I admit that it is very very unlikely. I guess it is just a wild dream of mine to run Java bytecodes and Python bytecodes on Python VM. I do have a wild vision that we can import java libraries (as jar files) into CPython. ML -- http://mail.python.org/mailman/listinfo/python-list
Tkinter Problems on MAC OS 10.3.9 python2.5
Hello folks, my name is Thomas, and I am new to this newsgroup. So first I want to say hello. :-) ...done! Now, I have a problem concerning my new python2.5 install. With python2.3 and 2.4, Tkinter was no problem. Now, when I try to import it, the folowing happens: iPimpG4:~/Python-Dev/examples profipimp$ python Python 2.5 (r25:51918, Sep 19 2006, 08:49:13) [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin Type help, copyright, credits or license for more information. import Tkinter Traceback (most recent call last): File stdin, line 1, in module File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk/Tkinter.py, line 38, in module import _tkinter # If this fails your Python may not be configured for Tk ImportError: dlcompat: dyld: /Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python can't open library: /Library/Frameworks/Tk.framework/Versions/8.4/Tk (No such file or directory, errno = 2) -- http://mail.python.org/mailman/listinfo/python-list
Re: How to make a python file to a DLL?
many_years_after schrieb: Any solution? http://wiki.python.org/moin/elmer -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy/scipy: error of correlation coefficient (clumpy data)
sturlamolden wrote: robert wrote: here the bootstrap test will as well tell us, that the confidence intervall narrows down by a factor ~sqrt(10) - just the same as if there would be 10-fold more of well distributed new data. Thus this kind of error estimation has no reasonable basis for data which is not very good. The confidence intervals narrows when the amount of independent data increases. If you don't understand why, then you lack a basic understanding of statistics. Particularly, it is a fundamental assumption in most statistical models that the data samples are IDENTICALLY AND INDEPENDENTLY DISTRIBUTED, often abbreviated i.i.d. And it certainly is assumed in this case. If you tell the computer (or model) that you have i.i.d. data, it will assume it is i.i.d. data, even when its not. The fundamental law of computer science also applies to statistics: shit in = shit out. If you nevertheless provide data that are not i.i.d., like you just did, you will simply obtain invalid results. The confidence interval concerns uncertainty about the value of a population parameter, not about the spread of your data sample. If you collect more INDEPENDENT data, you know more about the population from which the data was sampled. The confidence interval has the property that it will contain the unknown true correlation 95% of the times it is generated. Thus if you two samples WITH INDEPENDENT DATA from the same population, one small and one large, the large sample will generate a narrower confidence interval. Computer intensive methods like bootstrapping and asymptotic approximations derived analytically will behave similarly in this respect. However, if you are dumb enough to just provide duplications of your data, the computer is dumb enough to accept that they are obtained statistically independently. In statistical jargon this is called pseudo-sampling, and is one of the most common fallacies among uneducated practitioners. that duplication is just an extreme example to show my need: When I get the data, there can be an inherent filter/damping or other mean of clumping on the data which I don't know of beforehand. My model is basically linear (its a preparation step for ranking valuable input data for a classification task, thus for data reduction), just the degree of clumping in the data is unknown. Thus the formula for r is ok, but that bare i.i.d. formula for the error (1-r**2)/sqrt(n) (or bootstrap_test same) is blind on that. Statistical software doesn't prevent the practitioner from shooting himself in the leg; it actually makes it a lot easier. Anyone can paste data from Excel into SPSS and hit ANOVA in the menu. Whether the output makes any sense is a whole other story. One can duplicate each sample three or four times, and SPSS would be ignorant of that fact. It cannot guess that you are providing it with crappy data, and prevent you from screwing up your analysis. The same goes for NumPy code. The statistical formulas you type in Python have certain assumptions, and when they are violated the output is of no value. The more severe the violation, the less valuable is the output. The interesting task is probably this: to check for linear correlation but weight clumping of data somehow for the error estimation. If you have a pathological data sample, then you need to specify your knowledge in greater detail. Can you e.g. formulate a reasonable stochastic model for your data, fit the model parameters using the data, and then derive the correlation analytically? no, its too complex. Or its just: additional clumping/fractality in the data. Thus, linear correlation is supposed, but the x,y data distribution may have less than 2 dimensions. No better model. Think of such example: A drunken (x,y) 2D walker is supposed to walk along a diagonal, but he makes frequent and unpredictable pauses/slow motion. You get x,y coordinates in 1 per second. His speed and time pattern at all do not matter - you just want to know how well he keeps his track. ( My application data is even worse/blackbox, there is not even such model ) I am beginning to think your problem is ill defined because you lack a basic understanding of maths and statistics. For example, it seems you were confusing numerical error (rounding and truncation error) with statistical sampling error, you don't understand why standard errors decrease with sample size, you are testing with pathological data, you don't understand the difference between independent data and data duplications, etc. You really need to pick up a statistics textbook and do some reading, that's my advice. I think I understand all this very well. Its not on this level. The problem has also nothing to do with rounding, sampling errors etc. Of course the error ~1/sqrt(n) is the basic assumption - not what I not know, but what I complain about :-) (Thus I even guessed the dumb formula for r_err well
Useless thread about some useless statistics
I recently came across the program 'sloccount' that attempts to calculate the time and money that went into writing software. http://www.dwheeler.com/sloccount/ If ran on the source code of python 2.5 the results are the following: Total Physical Source Lines of Code (SLOC) = 689,996 Development Effort Estimate, Person-Years= 191.35 Estimated Average Number of Developers = 48.52 Total Estimated Cost to Develop = $ 25,848,213 Please credit this data as generated using David A. Wheeler's 'SLOCCount'. I'm wondering if the person-years and estimated average number of developers numbers are accurate or not - not that it's of any importance, just wondering. And about the 25 million dollars total price tag, well, um :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Data structure for ordered sequence
[EMAIL PROTECTED] wrote: Fredrik Lundh wrote: [EMAIL PROTECTED] wrote: I am looking for a data structure to hold rectangles in a 2d space. Please point me to any module which does these operations: Insert a rectangle into a particular co-ordinate. Get the rectangle/s right/left/above/below side to a particular rectangle. Get all the rectangles within a box Delete a particular rectangle. how many rectangles do you plan to store in this structure? /F Around 150 max And seeking/change frequency? fix dimensions? Probably for a GUI/mouse thing. =Not worth worring about a 2D-tree structure. A Python list ? :-) Insert/change coordinates at no costs and seek the list with a 3-liner and you are quite fast :-) Any sophisticated tree fumbling will cost more. Or at max use a 1D-btree thing or so for x0,x1 or a fix 16x16 array .. -robert -- http://mail.python.org/mailman/listinfo/python-list
Re: Will GPL Java eat into Python marketshare?
I do have a wild vision that we can import java libraries (as jar files) into CPython. Isnt this being achieved by Jython (python code using Java libraries), and in future by the Java scripting framework added into Java 6 ? -- http://mail.python.org/mailman/listinfo/python-list
Re: multi split function taking delimiter list
Paddy wrote:
Paddy wrote:
Paddy wrote:
[EMAIL PROTECTED] wrote:
Hi, I'm looking for something like:
multi_split( 'a:=b+c' , [':=','+'] )
returning:
['a', ':=', 'b', '+', 'c']
whats the python way to achieve this, preferably without regexp?
Thanks.
Martin
I resisted my urge to use a regexp and came up with this:
from itertools import groupby
s = 'apple=blue+cart'
[''.join(g) for k,g in groupby(s, lambda x: x in '=+')]
['apple', '=', 'blue', '+', 'cart']
For me, the regexp solution would have been clearer, but I need to
stretch my itertools skills.
- Paddy.
Arghhh!
No colon!
Forget the above please.
- Pad.
With colon:
from itertools import groupby
s = 'apple:=blue+cart'
[''.join(g) for k,g in groupby(s,lambda x: x in ':=+')]
['apple', ':=', 'blue', '+', 'cart']
- Pad.
Automatic grouping may or may not work as intended. If some subsets
should not be split, the solution raises a new problem.
I have been demonstrating solutions based on SE with such frequency of
late that I have begun to irritate some readers and SE in sarcastic
exaggeration has been characterized as the 'Solution of Everything'.
With some trepidation I am going to demonstrate another SE solution,
because the truth of the exaggeration is that SE is a versatile tool for
handling a variety of relatively simple problems in a simple,
straightforward manner.
test_string = 'a:=b+c: apple:=blue:+cart''
SE.SE (':\==/:\=/ +=/+/')(test_string).split ('/') # For repeats
the SE object would be assigned to a variable
['a', ':=', 'b', '+', 'c: apple', ':=', 'blue:', '+', 'cart']
This is a nuts-and-bolts approach. What you do is what you get. What you
want is what you do. By itself SE doesn't do anything but search and
replace, a concept without a learning curve. The simplicity doesn't
suggest versatility. Versatility comes from application techniques.
SE is a game of challenge. You know the result you want. You know
the pieces you have. The game is how to get the result with the pieces
using search and replace, either per se or as an auxiliary, as in this
case for splitting. That's all. The example above inserts some
appropriate split mark ('/'). It takes thirty seconds to write it up and
see the result. No need to ponder formulas and inner workings. If you
don't like what you see you also see what needs to be changed. Supposing
we should split single colons too, adding the corresponding substitution
and verifying the effect is a matter of another ten seconds:
SE.SE (':\==/:\=/ +=/+/ :=/:/')(test_string).split ('/')
['a', ':=', 'b', '+', 'c', ':', ' apple', ':=', 'blue', ':', '', '+',
'cart']
Now we see an empty field we don't like towards the end. Why?
SE.SE (':\==/:\=/ +=/+/ :=/:/')(test_string)
'a/:=/b/+/c/:/ apple/:=/blue/://+/cart'
Ah! It's two slashes next to each other. No problem. We de-multiply
double slashes in a second pass:
SE.SE (':\==/:\=/ +=/+/ :=/:/ | //=/')(test_string).split ('/')
['a', ':=', 'b', '+', 'c', ':', ' apple', ':=', 'blue', ':', '+', 'cart']
On second thought the colon should not be split if a plus sign follows:
SE.SE (':\==/:\=/ +=/+/ :=/:/ :+=:/+/ | //=/')(test_string).split ('/')
['a', ':=', 'b', '+', 'c', ':', ' apple', ':=', 'blue:', '+', 'cart']
No, wrong again! 'Colon-plus' should be exempt altogether. And no spaces
please:
SE.SE (':\==/:\=/ +=/+/ :=/:/ :+=:+ = |
//=/')(test_string).split ('/')
['a', ':=', 'b', '+', 'c', ':', 'apple', ':=', 'blue:+cart']
etc.
It is easy to get carried away and to forget that SE should not be used
instead of Python's built-ins, or to get carried away doing contextual
or grammar processing explicitly, which gets messy very fast. SE fills a
gap somewhere between built-ins and parsers.
Stream editing is not a mainstream technique. I believe it has the
potential to make many simple problems trivial and many harder ones
simpler. This is why I believe the technique deserves more attention,
which, again, may explain the focus of my posts.
Frederic
--
http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
Fredrik Tolf wrote: If this doesn't work, might there be some other way to run untrusted code that I haven't thought of (apart from using O/S-specific stuff like SECCOMD, of course). There was a module called rexec which tries to give you a restricted environment for executing code. But it seems, that it is not maintained anymore, because there were too much problems with it. It seems, that it is very complicated to get a restricted execution environment without losing too much of Pythons functionality. One question is, what you want to achieve. As another posting in this thread mentioned, you can't get around of denial of service attacks, even in restricted or trusted environments. So I assume, that what you want is something like a sandbox, where specific file actions (deleting files, access to specific part of the FS at all) and some other things can be restricted or forbidden. I think, this should be possible, even for some DOS-Attacks (e.g. restricting the amount of memory that can be used by the script, or the max stack size, depth of recursion limits etc.), but it is a hard job to find all places, where code can break out of your sandbox. For a full load of bad examples, simply have a look at JavaScript... For a IMHO really good implementation of the sandbox idea, have a look at the safe interp in Tcl. A short description (and by no mean complete) of the safe interp is to run a second and completely independent interpreter with all possibly dangerous commands completely removed and a one-way-channel to inject commands and scripts into its evaluation loop from the trusted interpreter. Depending on how much faith you have into the untrusted script, you can selectively allow additional commands in the safe interp or map common commands to other restricted or monitored versions of them, which you implemented yourself inside your trusted environment. I do not know, how complex it would be to do this in Python (since Tcl may look a little old fashioned to some people but has some unique features that helps especially with this kind of problem, such as having no keywords, which makes it possible to change the semantics of even the most basic constructs in the language from the scripting level), but I think it would be a really useful feature for Python to have a sandbox mechanism to run untrusted code. Regards Stephan -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml/ElementTree and .tail
Stefan Behnel wrote: [Remove an element, remove following nodes] Yes, it is. Just look at the API. It's an attribute of an Element, isn't it? What other API do you know where removing an element from a data structure leaves part of the element behind? I guess it depends on what you regard an element to be... [...] IMHO, DOM has a pretty significant mismatch with Python. ...in the DOM or otherwise: http://www.w3.org/TR/2006/REC-xml-20060816/#sec-logical-struct Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Py3K idea: why not drop the colon?
John Salerno wrote:
personally, i don't mind the colon and see no need to lose it, but if we
are talking in the realm of aesthetics, it actually seems like it would
be cleaner if it weren't there...sure, at first everyone who is used to
it might feel like something is missing, or the line is hanging open,
but overall the less characters, the better, right? isn't that why the
braces are gone?
Which is the better writing style for text intended for human readers?
I have some things to say:
Seagulls belong in the sky.
Colon fits in the text.
I have some things to say
{
Seagulls belong in the text.
Colon fits in the belly.
}
--
http://mail.python.org/mailman/listinfo/python-list
Re: Will GPL Java eat into Python marketshare?
Maurice LING wrote: I admit that it is very very unlikely. I guess it is just a wild dream of mine to run Java bytecodes and Python bytecodes on Python VM. I do have a wild vision that we can import java libraries (as jar files) into CPython. http://sourceforge.net/projects/jpype /F -- http://mail.python.org/mailman/listinfo/python-list
Re: Data structure for ordered sequence
robert wrote: how many rectangles do you plan to store in this structure? Around 150 max And seeking/change frequency? fix dimensions? Probably for a GUI/mouse thing. =Not worth worring about a 2D-tree structure. A Python list ? :-) Insert/change coordinates at no costs and seek the list with a 3-liner and you are quite fast :-) Any sophisticated tree fumbling will cost more. Or at max use a 1D-btree thing or so for x0,x1 or a fix 16x16 array .. thanks for writing the reply I didn't get around to write ;-) /F -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml/ElementTree and .tail
Paul Boddie wrote: Yes, it is. Just look at the API. It's an attribute of an Element, isn't it? What other API do you know where removing an element from a data structure leaves part of the element behind? I guess it depends on what you regard an element to be... Stefan said Element, not element. Element is a class in the ElementTree module, which can be used to *represent* an XML element in an XML infoset, including all the data *inside* the XML element, and any data *between* that XML element and the next one (which is always character data, of course). It's not very difficult, really; especially if you, as Stefan said, think in infoset terms rather a sequence of little piggies terms. /F -- http://mail.python.org/mailman/listinfo/python-list
split CSV fields
What is a most simple expression for splitting a CSV line with -protected fields? s='123,a,b,\c\,5.640' -- http://mail.python.org/mailman/listinfo/python-list
Re: Py3K idea: why not drop the colon?
Magnus Lycka wrote:
Which is the better writing style for text intended for human readers?
I have some things to say:
Seagulls belong in the sky.
Colon fits in the text.
I have some things to say
{
Seagulls belong in the text.
Colon fits in the belly.
}
have anyone counted the number of colons used in this way by the OP,
in his first few posts ? (if there isn't a name for the law that
states that the number for a let's remove the colons proposal is
always greater than zero, someone should come up with one...)
/F
--
http://mail.python.org/mailman/listinfo/python-list
Re: split CSV fields
s.split(',');
robert wrote:
What is a most simple expression for splitting a CSV line with -protected
fields?
s='123,a,b,\c\,5.640'
--
http://mail.python.org/mailman/listinfo/python-list
Re: Py3K idea: why not drop the colon?
Fredrik Lundh wrote:
Magnus Lycka wrote:
Which is the better writing style for text intended for human readers?
I have some things to say:
Seagulls belong in the sky.
Colon fits in the text.
I have some things to say
{
Seagulls belong in the text.
Colon fits in the belly.
}
have anyone counted the number of colons used in this way by the OP,
in his first few posts ? (if there isn't a name for the law that
states that the number for a let's remove the colons proposal is
always greater than zero, someone should come up with one...)
I'm always surprised by the amount of energy that's put into this type
of discussion - even the OP has suggested that this isn't a big issue.
If it's not a big issue why is this thread still going? Every language
has a syntax. Why not just accept it as a given and get on with more
productive activities?
regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden
--
http://mail.python.org/mailman/listinfo/python-list
Re: split CSV fields
robert wrote: What is a most simple expression for splitting a CSV line with -protected fields? s='123,a,b,\c\,5.640' import csv the preferred way is to read the file using that module. if you insist on processing a single line, you can do cols = list(csv.reader([string])) /F -- http://mail.python.org/mailman/listinfo/python-list
Re: split CSV fields
robert wrote: What is a most simple expression for splitting a CSV line with -protected fields? s='123,a,b,\c\,5.640' Use the csv-module. It should have a dialect for this, albeit I'm not 100% sure if the escaping of the is done properly from csv POV. Might be that it requires excel-standard. Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Py3K idea: why not drop the colon?
Steve Holden wrote: I'm always surprised by the amount of energy that's put into this type of discussion - even the OP has suggested that this isn't a big issue. If it's not a big issue why is this thread still going? http://www.bikeshed.org/ /F -- http://mail.python.org/mailman/listinfo/python-list
Re: Decimal() instead of float?
Terry Reedy wrote: John Salerno [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] John Machin wrote: Here in Austraila, (I expect this is common to most countries), there are people who are utterly clueless about elementary data model rules, like identification numbers should be kept as strings. Do you mean that ID numbers that serve as a primary key in a database should also be strings? If you mean user-entered data like social security, phone, account, part, or postal code 'numbers' -- as opposed to internal db-generated numbers that the user never sees -- this I would presume 'yes'. The modern trend is to use such values as alternate keys, and to have all tables use an automatically-allocated integer (autoincrement, identity, sequence) field as the primary key. Unfortunately some applications are getting such large tables that a 32-bit field is insufficient to enumerate all existing and deleted rows. Then you have to start keeping tables of unused primary keys. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden -- http://mail.python.org/mailman/listinfo/python-list
Re: Python v PHP: fair comparison?
bruce wrote: ummm bruno... you don't 'need' apache to run php. in fact, although i'm from the old hard c/c++ world way before web apps, i haven't really found much for most general apps (not ui/not threaded stuff) that php can't do.. You simply haven't been looking hard enough. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: pyfaq 1.0b1 is now available
John Machin wrote: P.S. Great job on the wiki, and yes a search facility a tad smarter than browser Ctrl-F would be a very good idea! the effbot.org version of the PyFAQ is basically just an editing and staging area; when everyone involved gets enough spare cycles, the plan is (or at least was) that it's going to be republished here: http://www.python.org/doc/faq/ and thus be covered by python.org's search engine, like everything else on that site. (but maybe one could use google's new community search, or whatever it's called, to set up a local search engine for the pyfaq/pyref/pytut zones. hmm. that's one more thing on my todo list...) /F -- http://mail.python.org/mailman/listinfo/python-list
Re: split CSV fields
robert wrote: What is a most simple expression for splitting a CSV line with -protected fields? s='123,a,b,\c\,5.640' import csv class mydialect(csv.excel): ... escapechar = \\ ... csv.reader(['123,a,b,\\c\\,5.640'], dialect=mydialect).next() ['123', 'a,b,c', '5.640'] Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Decimal() instead of float?
Steve Holden [EMAIL PROTECTED] writes: Unfortunately some applications are getting such large tables that a 32-bit field is insufficient to enumerate all existing and deleted rows. Then you have to start keeping tables of unused primary keys. Please tell me that's not from some Kafka nightmare. They don't use 64-bit ints? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python v PHP: fair comparison?
James Cunningham wrote: On 2006-11-15 20:59:26 -0500, walterbyrd [EMAIL PROTECTED] said: Bruno Desthuilliers wrote: walterbyrd a écrit : You mean there are web hosting companies that are still using Apache 1.3.x ? Practically all web-hosters still use Apache 1.3.x. Certainly all of the lower priced hosters. Not true. Dreamhost, at least, uses Apache 2. So you thin a single counter-example disproves an assertion that begins with practically all? C'mon, let's be serious. I just ordered a dedibox - a small dedicated web server - for my personal use. It's only 30 euros a month, you know... What if I only need 25mb of space? I can that with a PHP hoster for $10 a year. That is about 1/36th the price that you posting about. Maybe you don't mind paying 30 euros a month, but a lot others do. Of course, you can't get $10 a year plans on Dreamhost. But with deals you can get it down to about that low, at least for the first year. I suppose I'm lucky enough not to miss $8 a month otherwise. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml/ElementTree and .tail
Fredrik Lundh wrote: It's not very difficult, really; especially if you, as Stefan said, think in infoset terms rather a sequence of little piggies terms. Are piggies part of the infoset too? Does the Piggie class represent a piggie from the infoset plus a stretch of the road to the market? ;-) Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Py3K idea: why not drop the colon?
Fredrik Lundh wrote: Steve Holden wrote: I'm always surprised by the amount of energy that's put into this type of discussion - even the OP has suggested that this isn't a big issue. If it's not a big issue why is this thread still going? http://www.bikeshed.org/ I deliberately avoided using that analogy, but I'm sorry to say it *does* apply. I'd hate to see c.l.py descend to becoming a wibble group. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden -- http://mail.python.org/mailman/listinfo/python-list
Re: More puzzling behavior while subclassing datetime
Yes. Consider: def today(time=None, *args): ... print time = , time, args = , args ... today(2006, 11, 16) time = 2006 args = (11, 16) To fix the issue you'll probably have to remove the time=None parameter from GeneralizedTime.__new__() and instead extract it from args or kwargs. D'oh. That *should* have been obvious. I am now no longer allowed to program after midnight. Thanks! -Ben -- http://mail.python.org/mailman/listinfo/python-list
Re: Py3K idea: why not drop the colon?
Steve Holden wrote: y is this thread still going? http://www.bikeshed.org/ I deliberately avoided using that analogy, but I'm sorry to say it *does* apply. I'd hate to see c.l.py descend to becoming a wibble group. I switched to a reader that lets me kill threads with a single keypress a long time ago. which makes me wonder why I saw Magnus post and your followup. wait a second... -- http://mail.python.org/mailman/listinfo/python-list
Re: Will GPL Java eat into Python marketshare?
John Bokma wrote: Seriously though, there is no contradiction between the idea of people use Python instead of Java because they prefer to avoid pain It sounds like a typical fanboy statement to me, since it implies that Java is always a pain, and Python is perfect. That inference requires as preliminary convention, some clear contrafactual like everybody uses Python and nobody uses Java. IMO, this is a form of assumption that is only legitimate in the purpose of reaching agreement; while the OP's statement can be read as those people who use Python instead of Java do it because... without recourse to a similar assumption - what escapes your criticism. Cheers, BB -- http://mail.python.org/mailman/listinfo/python-list
Re: split CSV fields
Fredrik Lundh wrote: robert wrote: What is a most simple expression for splitting a CSV line with -protected fields? s='123,a,b,\c\,5.640' import csv the preferred way is to read the file using that module. if you insist on processing a single line, you can do cols = list(csv.reader([string])) /F Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32 | import csv | s='123,a,b,\c\,5.640' | cols = list(csv.reader([s])) | cols [['123', 'a,b,c', '5.640']] # maybe we need a bit more: | cols = list(csv.reader([s]))[0] | cols ['123', 'a,b,c', '5.640'] I'd guess that the OP is expecting 'a,b,c' for the second field. Twiddling with the knobs doesn't appear to help: | list(csv.reader([s], escapechar='\\'))[0] ['123', 'a,b,c', '5.640'] | list(csv.reader([s], escapechar='\\', doublequote=False))[0] ['123', 'a,b,c', '5.640'] Looks like a bug to me; AFAICT from the docs, the last attempt should have worked. Cheers, John -- http://mail.python.org/mailman/listinfo/python-list
Tkinter Python 2.5 Problems on MAC OS 10.3.9
Hello folks, Since this is my first post on the list, a brief introduction of myself. My name is Thomas, I am 26 years old, I am a student of Computational Linguistics and I am a python user. :-) Now my problem: I have Tcl/Tk 8.4.4 installed: iPimpG4:~ profipimp$ tclsh % info patchlevel 8.4.4 % But when I try to import Tkinter import Tkinter Traceback (most recent call last): File stdin, line 1, in module File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk/Tkinter.py, line 38, in module import _tkinter # If this fails your Python may not be configured for Tk ImportError: dlcompat: dyld: /Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python can't open library: /Library/Frameworks/Tk.framework/Versions/8.4/Tk (No such file or directory, errno = 2) ...this happens. Why? Tkinter worked perfectly with 2.3 and 2.4... Cheers, Thomas -- http://mail.python.org/mailman/listinfo/python-list
Re: split CSV fields
John Machin wrote: Fredrik Lundh wrote: robert wrote: What is a most simple expression for splitting a CSV line with -protected fields? s='123,a,b,\c\,5.640' import csv the preferred way is to read the file using that module. if you insist on processing a single line, you can do cols = list(csv.reader([string])) /F Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32 | import csv | s='123,a,b,\c\,5.640' | cols = list(csv.reader([s])) | cols [['123', 'a,b,c', '5.640']] # maybe we need a bit more: | cols = list(csv.reader([s]))[0] | cols ['123', 'a,b,c', '5.640'] I'd guess that the OP is expecting 'a,b,c' for the second field. Twiddling with the knobs doesn't appear to help: | list(csv.reader([s], escapechar='\\'))[0] ['123', 'a,b,c', '5.640'] | list(csv.reader([s], escapechar='\\', doublequote=False))[0] ['123', 'a,b,c', '5.640'] Looks like a bug to me; AFAICT from the docs, the last attempt should have worked. Given Peter Otten's post, looks like (1) there's a bug in the fmtparam mechanism -- it's ignoring the escapechar in my first twiddle, which should give the same result as Peter's. (2) | csv.excel.doublequote True According to my reading of the docs: doublequote Controls how instances of quotechar appearing inside a field should be themselves be quoted. When True, the character is doubled. When False, the escapechar is used as a prefix to the quotechar. It defaults to True. Peter's example should not have worked. -- http://mail.python.org/mailman/listinfo/python-list
Re: split CSV fields
John Machin wrote: Given Peter Otten's post, looks like (1) there's a bug in the fmtparam mechanism -- it's ignoring the escapechar in my first twiddle, which should give the same result as Peter's. (2) | csv.excel.doublequote True According to my reading of the docs: doublequote Controls how instances of quotechar appearing inside a field should be themselves be quoted. When True, the character is doubled. When False, the escapechar is used as a prefix to the quotechar. It defaults to True. Peter's example should not have worked. the documentation also mentions a quoting parameter that controls when quotes should be generated by the writer and recognised by the reader.. not sure how that changes things. anyway, it's either unclear documentation or a bug in the code. better submit a bug report so someone can fix one of them. /F -- http://mail.python.org/mailman/listinfo/python-list
Re: split CSV fields
John Machin wrote: John Machin wrote: Fredrik Lundh wrote: robert wrote: What is a most simple expression for splitting a CSV line with -protected fields? s='123,a,b,\c\,5.640' import csv the preferred way is to read the file using that module. if you insist on processing a single line, you can do cols = list(csv.reader([string])) /F Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32 | import csv | s='123,a,b,\c\,5.640' | cols = list(csv.reader([s])) | cols [['123', 'a,b,c', '5.640']] # maybe we need a bit more: | cols = list(csv.reader([s]))[0] | cols ['123', 'a,b,c', '5.640'] I'd guess that the OP is expecting 'a,b,c' for the second field. Twiddling with the knobs doesn't appear to help: | list(csv.reader([s], escapechar='\\'))[0] ['123', 'a,b,c', '5.640'] | list(csv.reader([s], escapechar='\\', doublequote=False))[0] ['123', 'a,b,c', '5.640'] Looks like a bug to me; AFAICT from the docs, the last attempt should have worked. Given Peter Otten's post, looks like (1) there's a bug in the fmtparam mechanism -- it's ignoring the escapechar in my first twiddle, which should give the same result as Peter's. (2) | csv.excel.doublequote True According to my reading of the docs: doublequote Controls how instances of quotechar appearing inside a field should be themselves be quoted. When True, the character is doubled. When False, the escapechar is used as a prefix to the quotechar. It defaults to True. Peter's example should not have worked. Doh. The OP's string was a raw string. I need some sleep. Scrap bug #1! | s=r'123,a,b,\c\,5.640' | list(csv.reader([s]))[0] ['123', 'a,b,\\c\\', '5.640'] # What's that??? | list(csv.reader([s], escapechar='\\'))[0] ['123', 'a,b,c', '5.640'] | list(csv.reader([s], escapechar='\\', doublequote=False))[0] ['123', 'a,b,c', '5.640'] And there's still the problem with doublequote Goodnight ... -- http://mail.python.org/mailman/listinfo/python-list
Re: split CSV fields
John Machin wrote: | s='123,a,b,\c\,5.640' Note how I fixed the input: '123,a,b,\c\,5.640' '123,a,b,c,5.640' '123,a,b,\\c\\,5.640' '123,a,b,\\c\\,5.640' Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: split CSV fields
Fredrik Lundh wrote: John Machin wrote: Given Peter Otten's post, looks like (1) there's a bug in the fmtparam mechanism -- it's ignoring the escapechar in my first twiddle, which should give the same result as Peter's. (2) | csv.excel.doublequote True According to my reading of the docs: doublequote Controls how instances of quotechar appearing inside a field should be themselves be quoted. When True, the character is doubled. When False, the escapechar is used as a prefix to the quotechar. It defaults to True. Peter's example should not have worked. the documentation also mentions a quoting parameter that controls when quotes should be generated by the writer and recognised by the reader.. not sure how that changes things. Hi Fredrik, I read that carefully -- quoting appears to have no effect in this situation. anyway, it's either unclear documentation or a bug in the code. better submit a bug report so someone can fix one of them. Tomorrow :-) Cheers, John -- http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
Fredrik Lundh wrote: timmy wrote: congraulations you have discovered loops and their misuse if you don't know what the phrase denial of service attack means, you can always google for it. /F maybe you should google linux kernel limit and you can prevent any user/process maxing out your system -- http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
Steven D'Aprano wrote: On Thu, 16 Nov 2006 17:44:37 +1000, timmy wrote: congraulations you have discovered loops and their misuse Did you have a point in your utterly inane comment, or did you just want to see your name on Usenet? In any case, it isn't just loops that are dangerous. print 2**512**512 No loop there, but it will operate as a lovely DoS attack if you run it. The Original Poster is suggesting running UNTRUSTED code. That means you have to assume that it will be actively hostile, but even if it isn't deliberately hostile, there will be bugs which the developer can't control. He wants to run this untrusted (hostile or buggy or both) code in an environment where it can't do bad things. Bad things include Denial of Service attacks. So, Timmy, let's hear your brilliant scheme for preventing DoS attacks when running hostile code in Python. as posted before, linux kernel limit. then you and your users can go as crazy as you want and you won't take out your system. maybe you should think a little more before going on the attack like that. -- http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
timmy wrote: maybe you should google linux kernel limit and you can prevent any user/process maxing out your system one would have thought that the phrase apart from OS-specific stuff might have meant that the OP wasn't asking for Linux-specific solutions. /F -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml/ElementTree and .tail
Thanks for the comments and thoughts. I must admit that I have an overwhelming feeling of having just stepped into the middle of a complex, heated conversation without having heard the preamble. (FYI, this reply is only an attempt to help those that come afterwards -- I'm not looking to advocate much of anything here.) Fredrik's invocation of the infoset term led me to a couple of quick searches that clarified the state of play. Here he sets the stage for the .tail behaviour that I originally posted about: http://effbot.org/zone/element-infoset.htm And it looks like there have been tussles over other mismatches in expectations before, specifically around how namespaces are handled: http://groups.google.com/group/comp.lang.python/browse_thread/thread/ 31b2e9f4a8f7338c http://nixforums.org/ntopic43901.html From what I can see, there are more than a few people that have stumbled with ElementTree's API because of their preexisting expectations, which others have probably correctly bucketed as implementation details. This comes as quite a shock to those who have stumbled (including myself) who have, lo these many years, come to view those details as the only standard that matters (perhaps simply because those details have been so consistent in our experience). Which, in my view, is just fine -- different strokes for different folks, and all that. When I originally started poking around the python xml world, I was somewhat confused as to why 4suite/Domlette existed, as it seemed pretty clear that ElementTree had crystallized a lot of mindshare, and has a very attractive API to boot. Thankfully, I can now see its appeal, and am very glad it's around, as it seems to have all of those comfortable implementation details that I've been looking for. :-) As for the infoset vs. sequence of piggies nut: if ElementTree's infoset approach is technically correct, then wouldn't it also be correct to use a .head attribute instead of a .tail attribute? Example: afirstbmiddle/blast/a might be represented as: Element a: head='', text='last' Element b: head='first', text='middle' If I'm wrong, just chalk it up to the fact that this is the first time I've ever looked at the Infoset spec, and I'm simply confused. If that IS a technically-valid way to represent the above xml fragment . . . then I guess I'll make sure to tread more carefully in the future around tools that work in infoset terms. For me, it turns out that sequences of piggies really are important, at least in contexts where XML is merely a means to an end (either because of the attractiveness of the toolsets or because we must cope with what we're provided as input) and where consistency with existing tools (like those that adhere to DOM level 2/3) and expectations are critical. I think this is what Paul was nodding towards with his original response to Stefan's response. Cheers, - Chas On Nov 16, 2006, at 5:11 AM, Fredrik Lundh wrote: Paul Boddie wrote: Yes, it is. Just look at the API. It's an attribute of an Element, isn't it? What other API do you know where removing an element from a data structure leaves part of the element behind? I guess it depends on what you regard an element to be... Stefan said Element, not element. Element is a class in the ElementTree module, which can be used to *represent* an XML element in an XML infoset, including all the data *inside* the XML element, and any data *between* that XML element and the next one (which is always character data, of course). It's not very difficult, really; especially if you, as Stefan said, think in infoset terms rather a sequence of little piggies terms. /F -- http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
timmy timothy at open-networks.net wrote: This sub-thread starts to become a flame-war, isn't it? Calm down, both of you... No need to fight, when only some ideas for a technical question are requested. as posted before, linux kernel limit. then you and your users can go as crazy as you want and you won't take out your system. The problem with linux kernel limits are, that they won't work really good on MacOSX and Windows... OTOH the idea is the right one, but the effect can be achieved inside of Python. Since Python does byte compile the code and the interpreter evaluates each byte code token in one evaluation step. The interpreter could be extended for such usecases to count and limit the number of evaluation steps allowed for untrusted script or methods in untrusted script as well as to limit the recursion depth or memory to be allocated. All those limits are managed by the interpreter for script code and hence can be limited for untrusted code by the interpreter. This also does not really make DoS impossible (what about C extensions? - maybe restricting import?). - As I said before in this thread, making a sandbox really secure is a hard job, and may need some serious changes in the Python interpreter, but AFAIK from Tcl, it is possible - and would be nice to have. Regards Stephan -- http://mail.python.org/mailman/listinfo/python-list
pyj file extension
I have two questions for the list: - what extension is pyj ? - is there a good python debugger that can inspect precompiled python files? Thank you :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
Fredrik Lundh wrote: timmy wrote: maybe you should google linux kernel limit and you can prevent any user/process maxing out your system one would have thought that the phrase apart from OS-specific stuff might have meant that the OP wasn't asking for Linux-specific solutions. /F sorry i didn't see that. cpu and memory limiting aren't specific to linux though, any NT system can also do it. the only effective way to prevent someone with access to a compiler from performing a local dos on your system is use each os's resource controls. there's no cross platform way to do this, since every system has vastly different methods of memory and cpu time handling. This looks like a case where he will just have to accept this as a trade off (security is always a trade off) -- http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
Stephan Kuhagen wrote: timmy timothy at open-networks.net wrote: This sub-thread starts to become a flame-war, isn't it? Calm down, both of you... No need to fight, when only some ideas for a technical question are requested. i'm not fighting, sometimes i can be a little terse for that i aplogise. as posted before, linux kernel limit. then you and your users can go as crazy as you want and you won't take out your system. The problem with linux kernel limits are, that they won't work really good on MacOSX and Windows... OTOH the idea is the right one, but the effect can be achieved inside of Python. Since Python does byte compile the code and the interpreter evaluates each byte code token in one evaluation step. The interpreter could be extended for such usecases to count and limit the number of evaluation steps allowed for untrusted script or methods in untrusted script as well as to limit the recursion depth or memory to be allocated. idunno sounds like a lot of trouble to engineer a solution that has already got a solution. all win NT systems have resource managment and i imagine OS X would as well?? Regards Stephan -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml/ElementTree and .tail
Chas Emerick wrote: might be represented as: Element a: head='', text='last' Element b: head='first', text='middle' sure, and you could use a text subtype instead that kept track of the elements above it, and let the elements be sequences of their siblings instead of their children, and perhaps stuff everything in a dictionary. such a construct would also be able to hold the same data, and be very hard to use in most normal situations. If I'm wrong, just chalk it up to the fact that this is the first time I've ever looked at the Infoset spec, and I'm simply confused. the Infoset spec *is* the essence of XML; if you don't realize that an XML document is just a serialization of a very simple data model, you're bound to be fighting with XML all the time. but ET doesn't implement the Infoset spec as it is, of course: it uses a *simplified* model, carefully optimized for the large percentage of all XML formats that simply doesn't use mixed content. if you're doing document-style processing, you sometimes need to add an extra assignment or two, but unless you're doing *only* document-style processing, ET's API gives you a net win. (and even if you're doing only document-style processing, ET's speed and memory footprint gives you a net win over most competing technologies). /F -- http://mail.python.org/mailman/listinfo/python-list
Re: Tkinter Python 2.5 Problems on MAC OS 10.3.9
which distribution of python you use ? I use the one of http://pythonmac.org/packages/py25-fat/index.html withoutproblem -- http://mail.python.org/mailman/listinfo/python-list
Re: pyj file extension
bituman wrote: I have two questions for the list: - what extension is pyj ? AFAIK custom to some game that uses python - but I guess you already know that. - is there a good python debugger that can inspect precompiled python files? You can decompyle them: http://www.crazy-compilers.com/decompyle/ But only up to python 2.3. No idea if that is sufficient if there is anything newer out there. diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Will GPL Java eat into Python marketshare?
I'm using python couse its clean, fast, fast to develop, easy, beauty, an alternative. Java its a mainstream lang, GPL or not... i think *sorry for my (por) english walterbyrd escreveu: Some think it will. Up untill now, Java has never been standard across different versions of Linux and Unix. Some think that is one reason that some developers have avoided Java in favor of Python. Now that Java has been GPL'd that might change. IMO: it won't make much difference. But I don't really know. -- http://mail.python.org/mailman/listinfo/python-list
Re: Will GPL Java eat into Python marketshare?
walterbyrd wrote: Some think it will. How so? Just because Java is GPL doesn't mean you can type less while coding in it. BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream()); is still like that GPL or no GPL, no? Up untill now, Java has never been standard across different versions of Linux and Unix. The last company I worked for has been delivering products on Windows, Linux, and Solaris for years. We've never had to resort to any platform-specific stuff with Java. Some think that is one reason that some developers have avoided Java in favor of Python. Now that Java has been GPL'd that might change. Becoming a better Java, maybe. Becoming more like Python? No way. I have high hopes for Jython though, Charles Nutter from JRuby has been having an ongoing discussion with the Jython developers in jython-dev. Interesting stuff! IMO: it won't make much difference. But I don't really know. -- http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
timmy timothy at open-networks.net wrote: count and limit the number of evaluation steps allowed for untrusted script or methods in untrusted script as well as to limit the recursion depth or memory to be allocated. idunno sounds like a lot of trouble to engineer a solution that has already got a solution. all win NT systems have resource managment and i imagine OS X would as well?? Sounds very likely, but does not solve the problem. With resource management on the OS level you can indeed set some important limits for untrusted scripts, but there are at least two drawbacks, which come to my mind (and maybe more, that I'm not aware of): 1. OS level can always only implement the lowest common denominator of all OS resource managements to be platform independent, which is a strong requirement, IMO. 2. If you want to exec a untrusted script from inside a trusted interpreter giving it a sandbox, then the sandbox has the same OS level restrictions as the first interpreter (except when started in a separate process, which makes communication between trusted and untrusted parts much more complicated). Also you can't have such a fine grained control over the untrusted execution environment at the OS level, e.g. limiting the recursion depth, which is a very important limit for secure interpreters. Limiting the stack on the OS level is for example no solution for this, because the byte code may behave completely different on the stack (and regarding hidden internal recursion) as what the toplevel Python code does (does anyone understand, what I'm talking about... - I think I just reached the recurion limit of my english capabilities), which means that you can't set deterministic restrictions for untrusted code in a platform independent manner at the OS level. - Managing all this in the interpreter would solve the problem, at the cost of implementing lots of resource management code. A good sandbox seems to be a real adventure with few survivors, as can be seen in the JavaScript-world. Regards Stephan -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml/ElementTree and .tail
On Nov 16, 2006, at 7:25 AM, Fredrik Lundh wrote: If I'm wrong, just chalk it up to the fact that this is the first time I've ever looked at the Infoset spec, and I'm simply confused. the Infoset spec *is* the essence of XML; if you don't realize that an XML document is just a serialization of a very simple data model, you're bound to be fighting with XML all the time. The principle and the practice diverge significantly in our neck of the woods. The current project involves consuming and making sense of extraordinarily (and typically unnecessarily) complex XHTML. Of course, as you say, those documents are still serializations of a simple data model, but the types of manipulations we do happen to butt up very uncomfortably with the way ET does things. but ET doesn't implement the Infoset spec as it is, of course: it uses a *simplified* model, carefully optimized for the large percentage of all XML formats that simply doesn't use mixed content. if you're doing document-style processing, you sometimes need to add an extra assignment or two, but unless you're doing *only* document-style processing, ET's API gives you a net win. (and even if you're doing only document- style processing, ET's speed and memory footprint gives you a net win over most competing technologies). Yeah, documents are all we do -- XML just happens to be a pleasant intermediate format, and something we need to consume. The notion of an nicely-formatted XML is entirely foreign to the work that we do -- in fact, our current focus is (in part) dragging decidedly unstructured data out of those XHTML documents (among other source formats) and putting them into a reasonable, useful structure. I took some time last night to bang out some functions that squeezed ET's model (via lxml) into doing what we need, and it ended up requiring a lot more BD than I like. At that point, I swung over to 4suite, which dropped into place quite nicely. *shrug* I guess we're just in the minority with regard to our API requirements -- we happen to live in the corner cases. I'm certainly glad to have made the detour on a different path for a bit though. - Chas -- http://mail.python.org/mailman/listinfo/python-list
Re: Python v PHP: fair comparison?
On 2006-11-16 05:46:45 -0500, Steve Holden [EMAIL PROTECTED] said: James Cunningham wrote: On 2006-11-15 20:59:26 -0500, walterbyrd [EMAIL PROTECTED] said: Bruno Desthuilliers wrote: walterbyrd a écrit : You mean there are web hosting companies that are still using Apache 1.3.x ? Practically all web-hosters still use Apache 1.3.x. Certainly all of the lower priced hosters. Not true. Dreamhost, at least, uses Apache 2. So you thin a single counter-example disproves an assertion that begins with practically all? Nope. It disproves your assertion that certainly all of the lower priced hosters use Apache 1.3. Certainly all. Certainly all. Best, James -- http://mail.python.org/mailman/listinfo/python-list
Re: Python v PHP: fair comparison?
Cameron Laird ha escrito: In article [EMAIL PROTECTED], Luis M. González [EMAIL PROTECTED] wrote: . . . Then look no further. Learn python and go kick php developers asses in the market place. There are thousands of php developers out there. Do you want to be just one more? I'd rather learn something newer, and much more powerful. . [more Python cheerleading] . . Perhaps it's timely to clarify the newer above: Guido made Python public in '89-90, and Rasmus showed PHP to others in '94-95. OK. But since when has python been considered a viable alternative for web development? As a generalp purpose language, it's older. But as a web development language, it's olnly when people started to look for the rails killer and many python alternatives started to come up (although Django has been in development for a long time before all this hype). I remember well, just a few months ago, there were many alternatives (remember subway?). -- http://mail.python.org/mailman/listinfo/python-list
Re: Python v PHP: fair comparison?
Luis M. González wrote: But as a web development language, it's olnly when people started to look for the rails killer and many python alternatives started to come up (although Django has been in development for a long time before all this hype). nah, people have built web stuff on Python for as long as we've had a web. /F -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml/ElementTree and .tail
Chas Emerick wrote: The principle and the practice diverge significantly in our neck of the woods. The current project involves consuming and making sense of extraordinarily (and typically unnecessarily) complex XHTML. wasn't your original complaint that ET didn't do the right thing when you removed elements from a mixed-content tree? (something than can be trivially handled with a 2-line helper function) why mutate the tree if all you want is to extract information from it? doesn't sound very efficient to me... /F -- http://mail.python.org/mailman/listinfo/python-list
Re: Tkinter Python 2.5 Problems on MAC OS 10.3.9
Thomas Ploch wrote: Hello folks, Since this is my first post on the list, a brief introduction of myself. My name is Thomas, I am 26 years old, I am a student of Computational Linguistics and I am a python user. :-) Now my problem: I have Tcl/Tk 8.4.4 installed: iPimpG4:~ profipimp$ tclsh % info patchlevel 8.4.4 % But when I try to import Tkinter import Tkinter Traceback (most recent call last): File stdin, line 1, in module File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk/Tkinter.py, line 38, in module import _tkinter # If this fails your Python may not be configured for Tk ImportError: dlcompat: dyld: /Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python can't open library: /Library/Frameworks/Tk.framework/Versions/8.4/Tk (No such file or directory, errno = 2) ...this happens. Why? Tkinter worked perfectly with 2.3 and 2.4... Cheers, Thomas Where is your installation of Tcl/Tk? It sounds like Python can't find it. Were you using the standard MacPython builds previously, or Unix-based builds from Fink or DarwinPorts? -- Kevin Walzer Code by Kevin http://www.codebykevin.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Python v PHP: fair comparison?
Luis M. González wrote: OK. But since when has python been considered a viable alternative for web development? Since the Bobo era (ca. 1997), but quite possibly before. Sure, you had to build your own mega-framework back then, but that's what a lot of people were doing anyway. As a generalp purpose language, it's older. But as a web development language, it's olnly when people started to look for the rails killer and many python alternatives started to come up (although Django has been in development for a long time before all this hype). I remember maintaining a long list of Web frameworks a few *years* ago. It's true that most of them didn't resemble the slick marketed package that you see with something like TurboGears, but you might be surprised how much you got with Webware back in 2001: http://www.webwareforpython.org/Docs/RelNotes-0.5.html I remember well, just a few months ago, there were many alternatives (remember subway?). Some people would like you to believe that they pioneered the mega-framework, amongst other things. Freely available documentation undermines such claims if you know where to look (and actually choose to do so). Paul -- http://mail.python.org/mailman/listinfo/python-list
Heap Memory
Hi my name is Bugra Cakir, I have a question. How can we increase heap memory or total memory Python interpreter will use in order to avoid memory problems ? -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml/ElementTree and .tail
On Nov 16, 2006, at 8:12 AM, Fredrik Lundh wrote: Chas Emerick wrote: The principle and the practice diverge significantly in our neck of the woods. The current project involves consuming and making sense of extraordinarily (and typically unnecessarily) complex XHTML. wasn't your original complaint that ET didn't do the right thing when you removed elements from a mixed-content tree? (something than can be trivially handled with a 2-line helper function) Yes, that was the initial issue, but the delta between Elements and DOM-style elements leads to other issues. There's no doubt that the needed helpers are simple, but all things being equal, not having to carry them around anywhere we're doing DOM manipulations is a big plus. why mutate the tree if all you want is to extract information from it? doesn't sound very efficient to me... Because we're far from doing anything that is regular or one-off in nature. We're systematizing the extraction of data from functionally unstructured content, and it's flatly necessary to normalize the XHTML into something that can be easily consumed by the processes we've built that can do that content-data extraction/conversion from plain text, XML, PDF, and now XHTML. Remember, corner cases. :-) - Chas -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml/ElementTree and .tail
Fredrik Lundh wrote: Stefan Behnel wrote: If you want to copy part of of removed element back into the tree, feel free to do so. and that can of course be done with a short helper function. Oh, and obviously with a custom Element class in lxml that does this automatically for you behind the scenes. http://codespeak.net/lxml/element_classes.html http://codespeak.net/lxml/element_classes.html#default-class-lookup Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
Stephan Kuhagen wrote: Sounds very likely, but does not solve the problem. With resource management on the OS level you can indeed set some important limits for untrusted scripts, but there are at least two drawbacks, which come to my mind (and maybe more, that I'm not aware of): 1. OS level can always only implement the lowest common denominator of all OS resource managements to be platform independent, which is a strong requirement, IMO. I think I understand what you intend to say here: that some kind of Python sandbox relying on operating system facilities can only depend on facilities implemented in all of the most interesting operating systems (which I once referred to as the big three, accompanied by howls of protest/derision). Yet just as people like to say that choosing a language is all about choosing the right tool for the job, shouldn't the choice of operating system be significant as well? If you're running a Try Python Web site, as some people were doing a few months ago, isn't it important to choose the right operating system as part of the right complete environment, instead of having the theoretical possibility of running it on something like RISC OS, yet having someone take your site down within seconds anyway? I don't know whether it's the same people who like to promote how well Python plays with everything else who also demand totally cross-platform solutions (if it doesn't work on Windows, we won't do it), but if so, I'd be interested in how they manage to reconcile these views. [...] A good sandbox seems to be a real adventure with few survivors, as can be seen in the JavaScript-world. Certainly, there are interesting directions to be taken with safe execution at the language and runtime levels, but as technologies like Java (in particular) have shown, it's possible for a project or a company to find itself focusing heavily on such strategies at the cost of readily available, mature technologies which might be good enough. The emergence of virtualisation as a commodity technology would suggest that sandboxing language runtimes isn't as fashionable as it was ten years ago. Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Py3K idea: why not drop the colon?
On 2006-11-16, Steve Holden [EMAIL PROTECTED] wrote: I'm always surprised by the amount of energy that's put into this type of discussion - even the OP has suggested that this isn't a big issue. If it's not a big issue why is this thread still going? Every language has a syntax. Why not just accept it as a given and get on with more productive activities? That's all very well if the language has a compromised and ad-hoc syntax. But since Python has a nice clean syntax, it makes me want to be associated with it by investing in its discussions. ;-) -- Neil Cerutti -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml/ElementTree and .tail
Chas Emerick wrote: the delta between Elements and DOM-style elements leads to other issues. There's no doubt that the needed helpers are simple, but all things being equal, not having to carry them around anywhere we're doing DOM manipulations is a big plus. Because we're far from doing anything that is regular or one-off in nature. We're systematizing the extraction of data from functionally unstructured content, and it's flatly necessary to normalize the XHTML into something that can be easily consumed by the processes we've built that can do that content-data extraction/conversion from plain text, XML, PDF, and now XHTML. Remember, corner cases. :-) Hmm, then I really don't get why you didn't just write a customised XHTML API on top of lxml's custom Element classes feature. Hiding XML language specific behaviour directly in the Element classes really helps in getting your code clean, especially in larger code bases. Stefan -- http://mail.python.org/mailman/listinfo/python-list
deciding what is a dir (folder) in a zip file
Hello, I'm trying to read .zip files and drop from the listing those files that are directories. I'm using the zipfile module. Does anyone know how I can I tell which files in the .zip are directories? I have looked around the net for the information but I cannot make it out: the pkzip specification hints that it is in the external file attribute but also hints that it is dependent on the platform on which the .zip was made (I get .zips from lots of platforms). The info-zip mailing list is unfortunately swamped by spam, so I can't ask there. Googling has increased my suspicion that it is not just a matter of seeing if the file name ends in '/' and that the relevant external file attribute seems to have different values for people from different platforms, so just experimenting on my platform doesn't seem to be a safe solution. (I could os.system(unzip +fn) and search the disk for directories but I'd rather not, for reasons I won't list.) I'd be grateful for any help, Jim -- http://mail.python.org/mailman/listinfo/python-list
Re: python-ldap and Python 2.5
Michael Ströder wrote: But this seems to help (tested on my local system): http://sourceforge.net/tracker/index.php?func=detailaid=1575329group_id=2072atid=102072 Released python-ldap 2.2.1 yesterday which contains this fix. Ciao, Michael. -- http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
as posted before, linux kernel limit. then you and your users can go as crazy as you want and you won't take out your system. maybe you should think a little more before going on the attack like that. You should maybe read a little bit more when making bold statements about the feasibility of a sandboxed _PYTHON_. The OP wrote: I was thinking that maybe it could be possible to load and run untrusted Python code, simply by loading it in a module with a modified version of __builtins__. Without any reachable function that do unsafe operations, code running from there shouldn't be able to do evil things. At least to me - and I presume pretty much everybody except you in this thread - this means that he is interested in executing arbitrary pieces of python code inside the interpreter, which comes from e.g. players who customize their in-game behavior of their avatars. Now how exactly does linux (or any other resource limiting technique on any OS) help here - killing the whole game server surely isn't a desirable solution when one player goes berserk, might it be intentionally or not. It is a recurring and pretty much understandable request on c.l.py to be able to do so - sometimes it arises in the disguise of killable threads. But unfortunately the solution doesn't seem to be as simple as one would wish. Diez -- http://mail.python.org/mailman/listinfo/python-list
fast kd-tree or octree implementations for python
Hi. I'm doing a FEM (Finite Elements) code in python. It uses a tetrahedral mesh to represent the geometry. For post-processing one specifies a list of 3D coordinates to calculate field values at, which requires the tet that contains a given point. Right now I'm brute-forcing it checking each tet for each point, which is very slow on large meshes since the number of points you are looking for also tend to increase with mesh size. It seems a kd-tree or octree data-structure will allow me to do lookups in O(logN) time at the cost of building the data structure in O(N*logN) time. I am looking for preferably a fast kd-tree implementation with a GPL-compatible license that is already wrapped for Python, but I'd be willing to make my own wrappers if needed. So far I've found CGAL - http://www.cgal.org and GTS -- The GNU Triangulated Surface Library - http://gts.sourceforge.net/ . CGAL has python wrappers but the tree code is under the QPL license (not GPL compat) while GTS doesn't come with ready-made python wrappers. What are other good choices? Thanks Neilen -- you know its kind of tragic we live in the new world but we've lost the magic -- Battery 9 (www.battery9.co.za) -- http://mail.python.org/mailman/listinfo/python-list
Re: Will GPL Java eat into Python marketshare?
walterbyrd wrote: Some think it will. Up untill now, Java has never been standard across different versions of Linux and Unix. Some think that is one reason that some developers have avoided Java in favor of Python. Now that Java has been GPL'd that might change. IMO: it won't make much difference. But I don't really know. In my opinion, GPLing Java won't have any immediate effect on who uses which for what kind of project. Longer term, however, I see a number of efforts to clean up some of Java's more egregious syntax problems. I've already seen one proposal, and there will undoubtedly be others. If any of them fly, they'll probably be folded into the base in a few years, and that will diminish the ease of use / large project divide between the two languages. John Roth -- http://mail.python.org/mailman/listinfo/python-list
Re: Decimal() instead of float?
Paul Rubin wrote: Steve Holden [EMAIL PROTECTED] writes: Unfortunately some applications are getting such large tables that a 32-bit field is insufficient to enumerate all existing and deleted rows. Then you have to start keeping tables of unused primary keys. Please tell me that's not from some Kafka nightmare. They don't use 64-bit ints? I don't believe SQL Server , for example, yet supports 64-bit identity values. If it does, then the version that at least one Python user has in use certainly doesn't. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden -- http://mail.python.org/mailman/listinfo/python-list
Re: Python v PHP: fair comparison?
James Cunningham wrote: On 2006-11-16 05:46:45 -0500, Steve Holden [EMAIL PROTECTED] said: James Cunningham wrote: On 2006-11-15 20:59:26 -0500, walterbyrd [EMAIL PROTECTED] said: Bruno Desthuilliers wrote: walterbyrd a écrit : You mean there are web hosting companies that are still using Apache 1.3.x ? Practically all web-hosters still use Apache 1.3.x. Certainly all of the lower priced hosters. Not true. Dreamhost, at least, uses Apache 2. So you thin a single counter-example disproves an assertion that begins with practically all? Nope. It disproves your assertion that certainly all of the lower priced hosters use Apache 1.3. Certainly all. Certainly all. Certainly. But not my assertion. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml/ElementTree and .tail
Paul Boddie wrote: It's not very difficult, really; especially if you, as Stefan said, think in infoset terms rather a sequence of little piggies terms. Are piggies part of the infoset too? Does the Piggie class represent a piggie from the infoset plus a stretch of the road to the market? ;-) no, they just appear in serialized XML. if you want concrete piggies, you have to wrap ET's iterparse function, or perhaps the XMLParser class. /F -- http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
Paul Boddie wrote: implement the lowest common denominator of all OS resource managements to be platform independent, which is a strong requirement, IMO. I think I understand what you intend to say here: that some kind of Python sandbox relying on operating system facilities can only depend on facilities implemented in all of the most interesting operating systems (which I once referred to as the big three, accompanied by howls of protest/derision Oberon, Plan 9 and AmigaOS...? ;-) ). Yet just as people like to say that choosing a language is all about choosing the right tool for the job, shouldn't the choice of operating system be significant as well? Yes, it should. But it isn't most times, I think. Often you have the situation to run a certain app e.g. on a OS that you can't simply exchange to your needs, for example the game server you mentioned, if this should run on an external host which is not maintained by you. Personally I would always prefer an OS independent solution, because it makes you more flexible. Some OS may be a good choice at a given time, but after you app has grown up, you may come to the decision to change the OS for some reason, but can't because you app depends on some of its specific features. Especially for apps written in a scripting language I would try to avoid that. If you're running a Try Python Web site, as some people were doing a few months ago, isn't it important to choose the right operating system as part of the right complete environment, instead of having the theoretical possibility of running it on something like RISC OS, yet having someone take your site down within seconds anyway? I don't know whether it's the same people who like to promote how well Python plays with everything else who also demand totally cross-platform solutions (if it doesn't work on Windows, we won't do it), but if so, I'd be interested in how they manage to reconcile these views. I'm afraid, we can't have a perfect world... But as I stated in another posting before, I think it is possible, to get a secure execution environment in a platform independent manner. The Tcl people did it and since I made myself already very unpopular at c.l.tcl by requesting some of Pythons goods for Tcl, I can do the same here by requesting some of Tcls good inventions for Python... ;-) The emergence of virtualisation as a commodity technology would suggest that sandboxing language runtimes isn't as fashionable as it was ten years ago. Virtual environments are a good choice for some of the tasks that were done with sandboxes in the past. But I'm afraid, that they are too huge for many problems. Imagine running an instance of a virtual machine on a mobile phone, or needing to execute some hundreds of them in parallel on a game server (or CGI) which itself runs on a virtual host at your webhoster, and of course none of them should be able to kill it's neighbours, so all of them need their own VM... phiu, that would need a really big iron. So the the idea of VMs _is_ a good one for certain situations, but the need for secure execution environments inside an interpreter remains. Regards Stephan -- http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
after reading all the mails on this thread, I have the following observations. I am relatively new to python at its development side but very old as far as using python is concerned. firstly, talking about gnu/linux there is no question about security. python, if at all it is non-secure wont harm a linux machine much any ways. secondly with OS like windows, things will be non-secure, no matter what you do. and it will be un stable and un secure no matter what language you use. how far then is python secured or non-secured in its absolute sence? I need to use python for a very mission critical project. may be I will also use zope. so I will like to know how far I can trust python for security in its absolute (platform independent ) sence? I mean running unwanted code at run-time etc. thanks. Krishnakant. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python v PHP: fair comparison?
On 2006-11-16 09:08:43 -0500, Steve Holden [EMAIL PROTECTED] said: James Cunningham wrote: On 2006-11-16 05:46:45 -0500, Steve Holden [EMAIL PROTECTED] said: James Cunningham wrote: On 2006-11-15 20:59:26 -0500, walterbyrd [EMAIL PROTECTED] said: Bruno Desthuilliers wrote: walterbyrd a écrit : You mean there are web hosting companies that are still using Apache 1.3.x ? Practically all web-hosters still use Apache 1.3.x. Certainly all of the lower priced hosters. Not true. Dreamhost, at least, uses Apache 2. So you thin a single counter-example disproves an assertion that begins with practically all? Nope. It disproves your assertion that certainly all of the lower priced hosters use Apache 1.3. Certainly all. Certainly all. Certainly. But not my assertion. regards Steve So I should be bothered to read the name of whom I'm replying to? Sheesh. ;) Best, James -- http://mail.python.org/mailman/listinfo/python-list
Re: Secure Python
Diez B. Roggisch wrote: [Multiplayer game servers] Now how exactly does linux (or any other resource limiting technique on any OS) help here - killing the whole game server surely isn't a desirable solution when one player goes berserk, might it be intentionally or not. A significant issue is the architecture of the server itself. Is a per-process solution acceptable or must everything happen in the same process with lots of threads (or microthreads)? Of course, there are games using lots of microthreads, although I'm not sure whether they also use lots of processes, too, and it has been asserted that having lots of operating system threads or processes is just too resource intensive, but I think it's especially worth considering the nature of the platform you're using and what it offers. Presumably, the original idea with UNIX-based systems was that you'd employ lots of processes in order to serve lots of customers, players, and so on, and there were companies like Internet service providers using precisely that one process per customer model in a naive fashion (until they exceeded the limit on simultaneous process identifiers in one particular case, I believe). Subsequent work focusing on throwing lots of threads into a single server-side container and then trying to isolate them from each other, all whilst running the container on a UNIX variant - a classic Java architectural pattern - seems like a curious distraction when one considers the strong portfolio of appropriate and readily available technologies that are left unused in the operating system of the deployment environment concerned. It is a recurring and pretty much understandable request on c.l.py to be able to do so - sometimes it arises in the disguise of killable threads. But unfortunately the solution doesn't seem to be as simple as one would wish. And this is where the hot topics collide: people want performant multitasking with lots of shared state (the global interpreter lock controversy) together with sandboxing so that the individual threads can't access most of that shared state (the restricted execution controversy). But it's like using a trip to meet the neighbours to justify a mission to the moon: you can visit the neighbours at a much lower cost with the vehicles you already have. I hear that various operating systems support better interprocess communication these days, but then we meet the third hot topic: why won't it work on Windows? Something has to give. Paul -- http://mail.python.org/mailman/listinfo/python-list
ini files and plugins
Hi, I've made a basic LaTeX file editor in wxPython, but now I wanted to add it some features : 1 - create a sort of ini file where I can put the user configuration that will load itself on the application startup ; 2 - a simple plugin system with python files ( maybe to add new langages, etc.) ; I'm looking for simple articles, modules or code snippets on these subjects; thanks, 6TooL9 -- http://mail.python.org/mailman/listinfo/python-list
Re: Python v PHP: fair comparison?
In article [EMAIL PROTECTED], Luis M. González [EMAIL PROTECTED] wrote: . . . Perhaps it's timely to clarify the newer above: Guido made Python public in '89-90, and Rasmus showed PHP to others in '94-95. OK. But since when has python been considered a viable alternative for web development? As a generalp purpose language, it's older. But as a web development language, it's olnly when people started to look for the rails killer and many python alternatives started to come up (although Django has been in development for a long time before all this hype). I remember well, just a few months ago, there were many alternatives (remember subway?). I appreciate your clarification. I can report back that we certainly move in different circles; I, for example, knew of people with multi-million-dollar budgets deciding on Python- based Web technology for *serious* applications in '96. Ruby 1.0, perhaps you'll recall, was a Christmas gift for 1996. For this and allied reasons, it didn't occur to me to regard Ruby as the senior Web development language among the two. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multithreaded C API Python questions
Svein Seldal wrote:
You seem to use the functions in very rude manner - wanting to force the GIL
around at lowest level.
Better forget the word GIL and think about acquiring and releasing Threads.
For each thread wanting to execute Python stuff there has to be a thread state
(ts). And then you have to enter/leave for executing Python by using the thread
state.
PyEval_InitThreads enters Python (incl. the GIL) first time.
To let Python free in main thread do
ts=PyEval_SaveThread()// or more easy: Py_BEGIN_ALLOW_THREADS
to reenter Python do simply:
PyEval_RestoreThread(ts) // or: Py_END_ALLOW_THREADS
Forget all the low level PyGIL... functions.
when you really want another c thread to enter Python first time with no thread
state existing through Python itsel (better make the thread in Python with
thread.start_new ?), then you have to do once
ts_cthread = PyThreadState_New(interp)
then enter Python in this thread:
PyEval_AcquireThread( ts_cthread )
to leave again:
PyEval_ReleaseThread( ts_cthread )
If you just loop the call_py_function_send() mainly in this thread you
probably better create the thread at all in Python and make the loop there. You
probably stick too much to C-level at any price :-) Probably you just do a
PyRun_ in main thread and then everything else in Python, and expose
C-parts for the thread-loop to Python as function (in other_py_inits) - where
in the c-function you probably have the usual
Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS bracket during time
consuming/system/reenter-endangered message stuff.
Robert
I think I've found the bug, but I need to confirm this behavior.
My findings is that if you use PyEval_InitThreads(), it is crucial to
release the GIL with PyEval_ReleaseThread() afterwards.
The API docs states that you can release the GIL with
PyEval_ReleaseLock() or PyEval_ReleaseThread() AFAICS.
http://docs.python.org/api/threads.html under docs of void
PyEval_InitThreads().
However, if I do use PyEval_ReleaseLock() it will crash shortly after
with Fatal Python error: ceval: tstate mix-up in a multithreaded C
environment.
If I use PyEval_ReleaseThread() to release the GIL, my app seems stable.
I release the GIL like this:
PyThreadState *pts = PyGILState_GetThisThreadState();
PyEval_ReleaseThread(pts);
Now, is this a feature or expected behavior in python (i.e. unclear API
documentation), or is this a bug in python itself?
Regards,
Svein
PS:
For reference I did something like this in pseudo-code:
Py_Initialize();
PyEval_InitThreads();
other_py_inits(); // Load py modules, etc.
PyEval_ReleaseLock();// -- MAKES THE APP UNSTABLE
create_c_thread();
PyGILState_STATE gstate;
gstate = PyGILState_Ensure();
call_py_function_main(); // Py main() wont return
PyGILState_Release(gstate);
And the main of the C thread function looks like this:
while(1)
{
PyGILState_STATE gstate;
gstate = PyGILState_Ensure();
call_py_function_send();
PyGILState_Release(gstate);
}
--
http://mail.python.org/mailman/listinfo/python-list
Re: ini files and plugins
tool69 wrote: 1 - create a sort of ini file where I can put the user configuration that will load itself on the application startup ; http://docs.python.org/lib/module-ConfigParser.html ? 2 - a simple plugin system with python files ( maybe to add new langages, etc.) ; http://effbot.org/zone/import-string.htm ? /F -- http://mail.python.org/mailman/listinfo/python-list
Re: Tkinter Python 2.5 Problems on MAC OS 10.3.9
Kevin Walzer schrieb: Thomas Ploch wrote: Hello folks, Since this is my first post on the list, a brief introduction of myself. My name is Thomas, I am 26 years old, I am a student of Computational Linguistics and I am a python user. :-) Now my problem: I have Tcl/Tk 8.4.4 installed: iPimpG4:~ profipimp$ tclsh % info patchlevel 8.4.4 % But when I try to import Tkinter import Tkinter Traceback (most recent call last): File stdin, line 1, in module File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk/Tkinter.py, line 38, in module import _tkinter # If this fails your Python may not be configured for Tk ImportError: dlcompat: dyld: /Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python can't open library: /Library/Frameworks/Tk.framework/Versions/8.4/Tk (No such file or directory, errno = 2) ...this happens. Why? Tkinter worked perfectly with 2.3 and 2.4... Cheers, Thomas Where is your installation of Tcl/Tk? It sounds like Python can't find it. Were you using the standard MacPython builds previously, or Unix-based builds from Fink or DarwinPorts? I built python from source using gcc 4.0.1, but I have solved it by installing a version of Tcl/TkAqua, now everything runs fine. Still I dont know why that has happened. The fink directories are in searchpath... Thanks for your help, Thomas -- http://mail.python.org/mailman/listinfo/python-list
ANN: python-ldap-2.2.1
Find a new release of python-ldap: http://python-ldap.sourceforge.net/ python-ldap provides an object-oriented API to access LDAP directory servers from Python programs. It mainly wraps the OpenLDAP 2.x libs for that purpose. Additionally it contains modules for other LDAP-related stuff (e.g. processing LDIF, LDAPURLs and LDAPv3 schema). Released 2.2.1 2006-11-15 Changes since 2.2.0: Modules/ * Fix for Python 2.5 free(): invalid pointer (see SF#1575329) * passwd() accepts None for arguments user, oldpw, newpw (see SF#1440151) Lib/ * ldif.LDIFWriter.unparse() now accepts instances of derived dict and list classes (see SF#1489898) -- http://mail.python.org/mailman/listinfo/python-list
Re: ini files and plugins
Fredrik Lundh a écrit : tool69 wrote: 1 - create a sort of ini file where I can put the user configuration that will load itself on the application startup ; http://docs.python.org/lib/module-ConfigParser.html ? 2 - a simple plugin system with python files ( maybe to add new langages, etc.) ; http://effbot.org/zone/import-string.htm ? /F Thanks Fredrik, I'll take a look. -- http://mail.python.org/mailman/listinfo/python-list
Re: Yield
Now that we're on the subject, what are the advantages of using generators over, say, list comprehensions or for loops? It seems to me that virtually all (I won't say everything) the examples I've seen can be done just as easily without using generators. For example, Fredrik's initial example in the post: a = [1,2,3] for i in a: print i ... 1 2 3 sum(a) 6 [str(i) for i in a] ['1', '2', '3'] Carsten mentioned that generators are more memory-efficient to use when dealing with large numbers of objects. Is this the main advantage of using generators? Also, in what other novel ways are generators used that are clearly superior to alternatives? Thanks in advance, Danny On Nov 16, 3:14 am, John Machin [EMAIL PROTECTED] wrote: On 16/11/2006 7:00 PM, Fredrik Lundh wrote: John Machin wrote: I would like to thanks Fredrik for his contribution to improve that. Call me crazy, but after an admittedly quick read, the version on the wiki seems to be about word for word with on the docs.python.org version. maybe he was thinking about the article I posted, or the FAQ link I posted in a followup: http://effbot.org/pyfaq/what-is-a-generator.htm which was based on my article and the glossary entry from the tutorial: http://effbot.org/pytut/glossary.htm#generatorQuite possibly. Could you please tell us how you think the wiki version is an improvement? an add comment link? direct URL:s for all concepts in the language? extensive hyperlinking? semantic retargetable markup?Yes, they're great. Maybe I was thinking about the text contents only :-) Cheers, John -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy/scipy: error of correlation coefficient (clumpy data)
robert wrote: Think of such example: A drunken (x,y) 2D walker is supposed to walk along a diagonal, but he makes frequent and unpredictable pauses/slow motion. You get x,y coordinates in 1 per second. His speed and time pattern at all do not matter - you just want to know how well he keeps his track. In which case you have time series data, i.e. regular samples from p(t) = [ x(t), y(t) ]. Time series have some sort of autocorrelation in the samples as well, which must be taken into account. Even tough you could weight each point by the drunkard's speed, a correlation or linear regression would still not make any sense here, as such analyses are based on the assumption of no autocorrelation in the samples or the residuals. Correlation has no meaning if y[t] is correlated with y[t+1], and regression has no meaning if the residual e[t] is correlated with the residual e[t+1]. A state space model could e.g. be applicable. You could estimate the path of the drunkard using a Kalman filter to compute a Taylor series expansion p(t) = p0 + v*t + 0.5*a*t**2 + ... for the path at each step p(t). When you have estimates for the state parameters s, v, and a, you can compute some sort of measure for the drunkard's deviation from his ideal path. However, if you don't have time series data, you should not treat your data as such. If you don't know how your data is generated, there is no way to deal with them correctly. If the samples are time series they must be threated as such, if they are not they should not. If the samples are i.i.d. each point count equally much, if they are not they do not. If you have a clumped data due to time series or lack of i.i.d., you must deal with that. However, data can be i.i.d. and clumped, if the underlying distribution is clumped. In order to determine the cause, you must consider how your data are generated and how your data are sampled. You need meta-information about your data to determine this. Matlab or Octave will help you with this, and it is certainly not a weakness of NumPy as you implied in your original post. There is no way to put magic into any numerical computation. Statistics always require formulation of specific assumptions about the data. If you cannot think clearly about your data, then that is the problem you must solve. -- http://mail.python.org/mailman/listinfo/python-list
Will subprocess eventually deprecate popen2 ?
I have a few Python programs that use popen2, and they work quite nicely and dependably, so I don't really have any reason to change them to use the new subprocess module...unless of course there any future plans to deprecate popen2. Is this something I will have to plan for ? -- http://mail.python.org/mailman/listinfo/python-list
