ANN: Leo 4.9 final released
Leo 4.9 final is now available at: http://sourceforge.net/projects/leo/files/ Leo is a text editor, data organizer, project manager and much more. See: http://webpages.charter.net/edreamleo/intro.html The highlights of Leo 4.9: -- - The Qt gui completely replaces the Tk gui--Qt gui now has all essential features of Tk, including plugins. - Completed Leo's autocompleter. - The rendering pane can now display movies, html, svg images, etc. - The scrolledmessage plugin can now use the rendering pane. - Nodes may contain multiple @language directives. - Leo highlights URL's everywhere. Ctrl-click URL's opens them in your web browser. - Leo uses an @file node's extension by default if there is no @language directive in effect. - Unified extract and import commands. - Leo can now colorize multiple @language directives in a single node. - Plain up/down arrow keys in headline-editing mode select a new node. - New commands to insert, delete, change or print uA's. - Added namespace directive to .leo files. - Fixed many bugs, some important, most quite minor. Links: -- Leo: http://webpages.charter.net/edreamleo/front.html Forum:http://groups.google.com/group/leo-editor Download: http://sourceforge.net/projects/leo/files/ Bzr: http://code.launchpad.net/leo-editor/ Quotes: http://webpages.charter.net/edreamleo/testimonials.html Edward K. Ream June 21, 2011 -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[ANN] Shed Skin 0.8
Hi all, I have just released version 0.8 of Shed Skin, an experimental (restricted-)Python-to-C++ compiler. Please see my blog for the full announcement: http://shed-skin.blogspot.com The Shed Skin homepage can be found here: http://shedskin.googlecode.com Thanks, Mark Dufour -- http://www.youtube.com/watch?v=E6LsfnBmdnk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
gcc python plugin and static analyser for CPython sources
I've been working on a new plugin for GCC, which supports embedding
Python within GCC, exposing GCC's internal data structures as Python
objects and classes.
The plugin links against libpython, and (I hope) allows you to invoke
arbitrary Python scripts from inside a compile. My aim is to allow
people to write GCC plugins as Python scripts, and to make it much
easier to prototype new GCC features.
As an example of use for the plugin, I have also been writing a static
analysis tool for checking the C code of CPython extension modules. So
far this only checks the arguments passed to PyArg_ParseTuple*, but I'm
working on autodetecting reference counting errors, and turning these
into compile-time warnings (See [1])
The plugin is Free Software, licensed under the GPLv3 (or later).
The code can be seen here:
http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=summary
and the website for the plugin is the Trac instance here:
https://fedorahosted.org/gcc-python-plugin/
The documentation is in the docs subdirectory (using sphinx). You can
see a pre-built HTML version of the docs here:
http://readthedocs.org/docs/gcc-python-plugin/en/latest/index.html
It's still at the experimental proof-of-concept stage; expect crashes
and tracebacks.
However, it is already possible to use this to add additional compiler
errors/warnings, e.g. domain-specific checks, or static analysis.
One of my goals for this is to teach GCC about the common mistakes
people make when writing extensions for CPython [1], but it could be
used
- e.g. to teach GCC about GTK's reference-counting semantics,
- to check locking in the Linux kernel
- to check signal-safety in APIs, etc
- rapid prototyping
Other ideas include visualizations of code structure. There are handy
methods for plotting control flow graphs (using graphviz), showing the
source code interleaved with GCC's internal representation, such as the
one here:
http://readthedocs.org/docs/gcc-python-plugin/en/latest/cfg.html
It could also be used to build a more general static-analysis tool.
The CPython API checker has the beginnings of this:
Example output:
test.c: In function ‘leaky’:
test.c:21:10: error: leak of PyObject* reference acquired at call to
PyList_New at test.c:21 [-fpermissive]
test.c:22: taking False path at if (!list)
test.c:24: reaching here item = PyLong_FromLong(42);
test.c:27: taking True path at if (!item)
test.c:21: returning NULL
Numerous caveats right now (e.g. how I deal with loops is really
dubious). It's disabled for now within the source tree (I need to fix
my selftests to pass again...) It perhaps could be generalized to do
e.g. {malloc,FILE*, fd} leaks, array bounds checking, int overflow, etc,
but obviously that's a far bigger task.
So far, I'm just doing a limited form of abstract interpretation (or,
at least, based on my understanding of that term), dealing with explicit
finite prefixes of traces of execution, tracking abstract values (e.g.
NULL-ptr vs non-NULL-ptr) and stopping when the trace loops (which is
just an easy way to guarantee termination, not a good one, but for my
use-case is good enough, I hope. Plus it ought to make it easier to
generate highly-readable error messages).
Thanks to Red Hat for allowing me to devote a substantial chunk of
$DAYJOB to this over the last couple of months.
I hope this will be helpful to both the GCC and Python communities.
Dave
[1] see
http://readthedocs.org/docs/gcc-python-plugin/en/latest/cpychecker.html
and
https://fedoraproject.org/wiki/Features/StaticAnalysisOfCPythonExtensions
--
http://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations/
Harmattan Python is available
The Harmattan Python team is proud to announce the availability of a full-featured Python environment for the MeeGo 1.2 Harmattan platform. Supported on Nokia's N9 phone and preloaded on the Nokia N950 developer device as well as the Harmattan Platform SDK, the Harmattan Python environment allows you to write Python applications with complete device functionality and flawless user experience. You can even publish your Python applications on Ovi Store! The Harmattan Python environment is built on top of PySide: Python for Qt, which provides the developers complete Python bindings for the Qt Developer Frameworks, including seamless support for Qt Quick and Qt Components. Qt Mobility is fully supported as well, together with a representative set of MeeGo Harmattan APIs. In addition to these, a comprehensive set of essential Python modules are made available for the developers. This release is a culmination of several years of hard work and offers the most complete and full-featured Python programming language support on any mobile platform. Harmattan Python comes batteries included! See our Harmattan Python home page and start hacking! http://wiki.meego.com/Harmattan_Python -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Re: parse date string having EDT
Junaid P V junu...@gmail.com wrote:
I was trying to parse a date string containing EDT time zone
eg: 'Mon Jun 20 14:00:57 EDT 2011'
I tried:
datetime.strptime('Mon Jun 20 14:00:57 EDT 2011', '%a %b %d %H:%M:%S %Z %Y')
But I get error
Right, because strptime doesn't support %Z. You'll have to handle that
yourself and remove it before conversion.
--
Tim Roberts, t...@probo.com
Providenza Boekelheide, Inc.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Missing python27.dll on Win 7 64-bit
Hi! In my Win è 64 bits, python27.dll (and others DLLs like pythoncom27.dll, pythoncomloader27.dll or pywintypes27.dll) are in C:\Windows\SysWOW64 And (my) Python 2.7.2 run perfectly. @-salutations -- Michel Claveau -- http://mail.python.org/mailman/listinfo/python-list
Re: parse date string having EDT
Tim Roberts t...@probo.com writes: Right, because strptime doesn't support %Z. Au contraire: Support for the %Z directive is based on the values contained in tzname and whether daylight is true. Because of this, it is platform-specific except for recognizing UTC and GMT which are always known (and are considered to be non-daylight savings timezones). URL:http://docs.python.org/library/time.html#time.strptime -- \“Human reason is snatching everything to itself, leaving | `\ nothing for faith.” —Saint Bernard, 1090–1153 | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Rant on web browsers
Regarding pyjamas lib size, its actually not that big if you want only bare python, without DOM wrapper. You only need pyjslib, which is less than 30kb gzipped when compiled even with --strict (quite verbose mode). Even that could be further reduced if you drop unused by your code python features from it. -- With best regards, Daniel Kluev -- http://mail.python.org/mailman/listinfo/python-list
Re: Python scoping
On 21 Giu, 06:06, Ben Finney ben+pyt...@benfinney.id.au wrote: Chris Angelico ros...@gmail.com writes: On Tue, Jun 21, 2011 at 12:38 PM, Ben Finney ben+pyt...@benfinney.id.au wrote: The *binding* is scoped. And the binding follows the exact same rules as anything else would. It has scope and visibility. In terms of the OP, the binding IS like a variable. Yes. So let's stop behaving as though the *name* behaves like a variable. It isn't, and doesn't. -- \ “Computer perspective on Moore's Law: Human effort becomes | `\ twice as expensive roughly every two years.” —anonymous | _o__) | Ben Finney Ok, thanks for the clarification, I'll take extra care e.g. when dealing with exceptions. Ciao, Mattia -- http://mail.python.org/mailman/listinfo/python-list
Re: something about performence
I just wrote something. I could not run a profiler or analyze the timing but
I felt it was effecient. Have a look and see if it helps:
from itertools import *
def sep_add(line1, line2):
if line1 and line2:
val1 = line1.split()
val2 = line2.split()
if (val1 and val2) and (len(val1) == len(val2) == 2):
return (val1[0] == val2[0])
def add_col_files(file1, file2):
fs1 = open(file1, r)
fs2 = open(file2, r)
fsn = open(new_sample.txt, w)
# Zip the files together and find the ones that match the index
# process the tuple accordingly
# output should be what you want to write to the file
for k in ifilter(lambda (i,j): (sep_add(i,j)), izip(fs1, fs2)):
if k:
output = k[0] + + k[1] + \n # sample output
fsn.write(output)
fsn.close()
fs1.close()
fs2.close()
if __name__ == __main__:
import time
start = time.localtime(time.time())
print time.asctime(start)
#add_col_files(sample1.txt, sample3.txt)
end = time.localtime(time.time())
print time.asctime(end)
It took about a minute on my comp for comparing about 50-100M sized files.
As I said not done too much of testing on this code.
2011/6/21 Ken Seehart k...@seehart.com
**
On 6/20/2011 10:31 PM, Ken Seehart wrote:
On 6/20/2011 7:59 PM, king6c...@gmail.com wrote:
Hi，
I have two large files,each has more than 2 lines,and each line
consists of two fields,one is the id and the other a value,
the ids are sorted.
for example:
file1
(uin_a y)
1 1245
2 12333
3 324543
5 3464565
file2
(uin_b gift)
1 34545
3 6436466
4 35345646
5 463626
I want to merge them and get a file,the lines of which consists of an id
and the sum of the two values in file1 and file2。
the codes are as below:
uin_y=open('file1')
uin_gift=open(file2')
y_line=uin_y.next()
gift_line=uin_gift.next()
while 1:
try:
uin_a,y=[int(i) for i in y_line.split()]
uin_b,gift=[int(i) for i in gift_line.split()]
if uin_a==uin_b:
score=y+gift
print uin_a,score
y_line=uin_y.next()
gift_line=uin_gift.next()
if uin_auin_b:
print uin_a,y
y_line=uin_y.next()
if uin_auin_b:
print uin_b,gift
gift_line=uin_gift.next()
except StopIteration:
break
the question is that those code runs 40+ minutes on a server(16 core,32G
mem),
the time complexity is O(n),and there are not too much operations,
I think it should be faster.So I want to ask which part costs so much.
I tried the cProfile module but didn't get too much.
I guess maybe it is the int() operation that cost so much,but I'm not sure
and don't know how to solve this.
Is there a way to avoid type convertion in Python such as scanf in C?
Thanks for your help ：）
Unfortunately python does not have a scanf equivalent AFAIK. Most use cases
for scanf can be handled by regular expressions, but that would clearly
useless for you, and just slow you down more since it does not perform the
int conversion for you.
Your code appears to have a bug: I would expect that the last entry will be
lost unless both files end with the same index value. Be sure to test your
code on a few short test files.
I recommend psyco to make the whole thing faster.
Regards,
Ken Seehart
Another thought (a bit of extra work, but you might find it worthwhile if
psyco doesn't yield a sufficient boost):
Write a couple filter programs to convert to and from binary data (pairs of
32 or 64 bit integers depending on your requirements).
Modify your program to use the subprocess module to open two instances of
the binary conversion process with the two input files. Then pipe the output
of that program into the binary to text filter.
This might turn out to be faster since each process would make use of a
core. Also it gives you other options, such as keeping your data in binary
form for processing, and converting to text only as needed.
Ken Seehart
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: those darn exceptions
On Tue, 21 Jun 2011 01:43:39 +, Chris Torek wrote: Exceptions are great, but... Sometimes when calling a function, you want to catch some or even all the various exceptions it could raise. What exceptions *are* those? [snip much, much interactive code] TL;DR *wink* Shorter version: Is there any definitive list of what exceptions a function could raise? Shorter answer: No. [...] But how can I know a priori that os.kill() could raise OverflowError in the first place? You can't. Even if you studied the source code, you couldn't be sure that it won't change in the future. Or that somebody will monkey-patch os.kill, or a dependency, introducing a new exception. More importantly though, most functions are reliant on their argument. You *cannot* tell what exceptions len(x) will raise, because that depends on what type(x).__len__ does -- and that could be anything. So, in principle, any function could raise any exception. [...] Functions have a number of special __ attributes. I think it might be reasonable to have all of the built-in functions, at least, have one more, perhaps spelled __exceptions__, that gives you a tuple of all the exceptions that the function might raise. Imagine, then: Or the author of the function could document the exceptions that it raises. Either way, nothing prevents this list from getting out of sync with the reality of which exceptions could be raised. Another question -- is the list of exceptions part of the function's official API? *All* of the exceptions listed, or only some of them? Apart from your pylint example below -- which I don't find convincing in the least, see further comments later -- I don't see the point of this. You shouldn't have the attitude that If a function could raise an exception, I'm going to catch it. You have to understand the circumstances that a function might raise, and decide whether or not you want it caught. Hint: almost always, the answer is you don't. Either way, a mere list of exceptions doesn't give you much. This adds additional burden on the developer of the function, while giving little benefit to the user. os.kill.__doc__ 'kill(pid, sig)\n\nKill a process with a signal.' [this part exists] os.kill.__exceptions__ (type 'exceptions.OSError', type 'exceptions.TypeError', type 'exceptions.OverflowError', type 'exceptions.DeprecationWarning') [this is my new proposed part] With something like this, a pylint-like tool could compute the transitive closure of all the exceptions that could occur in any function, by using __exceptions__ (if provided) or recursively finding exceptions for all functions called, and doing a set-union. In general, you can't do this at compile-time, only at runtime. There's no point inspecting len.__exceptions__ at compile-time if len is a different function at runtime. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: break in a module
On 18Jun2011 03:50, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: | On Sat, 18 Jun 2011 12:36:42 +1000, Cameron Simpson wrote: | Just to throw another approach into the mix (because I was thinking | about the finally word), what about: | |raise StopImport | | along the lines of generators' raise StopIteration. | | Then the import machinery can catch it, no new keyword is needed and no | existing keyword needs feature creeping. | | The only problem is that the importing module needs to catch it, or else | you get a traceback. The importer shouldn't need to care what goes in | inside the module. I was thinking the import mechanism itself would catch it, not the user of the import statement. Just as this: for i in iterator: ... quietly ceases the loop when the iterator raises StopIteration, the importer would consider a module that raised StopImport during the import to have finished its import successfully. So the caller does an: import foo as normal, with no special wrapping. And the module goes: spam() if condition: raise StopIteration ham() cheese() Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ In article 323c4db9.6...@ss1.csd.sc.edu, lhart...@ss1.csd.sc.edu wrote: | It still is true that the best touring bike is the one that you are | riding right now. Anything can be used for touring. As long as you | can travel, you are touring. I beleive such true and profound statements are NOT allowed to be posted in this newsgroup, and are also against the charter. You've been warned. - Randy Davis DoD #0013 ra...@agames.com in rec.moto -- http://mail.python.org/mailman/listinfo/python-list
Re: How to iterate on a changing dictionary
Terry Reedy wrote: Other situations will need other solutions. Like a job's completion list. Some number of workers get a job, and by time the caller sould know who and what has finished. Then a dictionary would hold number of remaining jobs. Similar a downloading list. -- goto /dev/null -- http://mail.python.org/mailman/listinfo/python-list
Re: something about performence
On 6/20/2011 11:56 PM, king6c...@gmail.com wrote:
Thanks for your reply,Ken :)
I found the two files ends with the same id……so I am lazy^-^
I tried psyco,and unfortunately it costs nearly the same time as before.
Is it true that we can only get str from files in Python?
Nope^_* . There are many applications such as image processing that
involve working with binary data.
(^_* well, technically yes actually: read() does in fact return str, but
the str can contain binary data)
But in order to do this, you need to use any of several modules that
allow python to operate on flat data.
Two standard modules exist for this purpose: *array *and *struct*. In
addition there are others such as *numpy *(for mathematical
applications) and *ctypes *(for interoperability between python and C/C++).
For your application, the *struct *module is sufficient.
fout = open('junk.dat', 'wb') # open for writing binary
fout.write(struct.pack('LL', 123,234))
fout.write(struct.pack('LL', 123,234))
fout.write(struct.pack('LL', 3,4))
fout.close()
fin = open('junk.dat', 'rb') # open for reading binary
print struct.unpack('LL', fin.read(8))
(123, 234)
print struct.unpack('LL', fin.read(8))
(123, 234)
print struct.unpack('LL', fin.read(8))
(3, 4)
print struct.unpack('LL', fin.read(8)) # raises struct.error at end
of file (because 0 bytes were read)
Traceback (most recent call last):
File string, line 1, in fragment
struct.error: unpack requires a string argument of length 8
在 2011年6月21日 下午1:50，Ken Seehart k...@seehart.com
mailto:k...@seehart.com写 道：
On 6/20/2011 10:31 PM, Ken Seehart wrote:
On 6/20/2011 7:59 PM, king6c...@gmail.com
mailto:king6c...@gmail.com wrote:
Hi，
I have two large files,each has more than 2 lines,and
each line consists of two fields,one is the id and the other a
value,
the ids are sorted.
for example:
file1
(uin_a y)
1 1245
2 12333
3 324543
5 3464565
file2
(uin_b gift)
1 34545
3 6436466
4 35345646
5 463626
I want to merge them and get a file,the lines of which consists
of an id and the sum of the two values in file1 and file2。
the codes are as below:
uin_y=open('file1')
uin_gift=open(file2')
y_line=uin_y.next()
gift_line=uin_gift.next()
while 1:
try:
uin_a,y=[int(i) for i in y_line.split()]
uin_b,gift=[int(i) for i in gift_line.split()]
if uin_a==uin_b:
score=y+gift
print uin_a,score
y_line=uin_y.next()
gift_line=uin_gift.next()
if uin_auin_b:
print uin_a,y
y_line=uin_y.next()
if uin_auin_b:
print uin_b,gift
gift_line=uin_gift.next()
except StopIteration:
break
the question is that those code runs 40+ minutes on a server(16
core,32G mem),
the time complexity is O(n),and there are not too much operations,
I think it should be faster.So I want to ask which part costs so
much.
I tried the cProfile module but didn't get too much.
I guess maybe it is the int() operation that cost so much,but
I'm not sure
and don't know how to solve this.
Is there a way to avoid type convertion in Python such as scanf
in C?
Thanks for your help ：）
Unfortunately python does not have a scanf equivalent AFAIK. Most
use cases for scanf can be handled by regular expressions, but
that would clearly useless for you, and just slow you down more
since it does not perform the int conversion for you.
Your code appears to have a bug: I would expect that the last
entry will be lost unless both files end with the same index
value. Be sure to test your code on a few short test files.
I recommend psyco to make the whole thing faster.
Regards,
Ken Seehart
Another thought (a bit of extra work, but you might find it
worthwhile if psyco doesn't yield a sufficient boost):
Write a couple filter programs to convert to and from binary data
(pairs of 32 or 64 bit integers depending on your requirements).
Modify your program to use the subprocess module to open two
instances of the binary conversion process with the two input
files. Then pipe the output of that program into the binary to
text filter.
This might turn out to be faster since each process would make use
of a core. Also it gives you other options, such as keeping your
data in binary form for processing, and converting to text only as
needed.
Ken Seehart
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: basic bytecode to machine code compiler (part 3)
Rouslan Korneychuk wrote: if i != pindex: (less if x = pivot else greater).append(x) Just curious, is there a reason why you wrote this last line that way instead of using a normal if/else clause? Cheers! Uli -- Domino Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 -- http://mail.python.org/mailman/listinfo/python-list
ANN: Leo 4.9 final released
Leo 4.9 final is now available at: http://sourceforge.net/projects/leo/files/ Leo is a text editor, data organizer, project manager and much more. See: http://webpages.charter.net/edreamleo/intro.html The highlights of Leo 4.9: - The Qt gui completely replaces the Tk gui--Qt gui now has all essential features of Tk, including plugins. - Completed Leo's autocompleter. - The rendering pane can now display movies, html, svg images, etc. - The scrolledmessage plugin can now use the rendering pane. - Nodes may contain multiple @language directives. - Leo highlights URL's everywhere. Ctrl-click URL's opens them in your web browser. - Leo uses an @file node's extension by default if there is no @language directive in effect. - Unified extract and import commands. - Leo can now colorize multiple @language directives in a single node. - Plain up/down arrow keys in headline-editing mode select a new node. - New commands to insert, delete, change or print uA's. - Added namespace directive to .leo files. - Fixed many bugs, some important, most quite minor. Links: -- Leo: http://webpages.charter.net/edreamleo/front.html Forum:http://groups.google.com/group/leo-editor Download: http://sourceforge.net/projects/leo/files/ Bzr: http://code.launchpad.net/leo-editor/ Quotes: http://webpages.charter.net/edreamleo/testimonials.html Edward K. Ream June 21, 2011 -- http://mail.python.org/mailman/listinfo/python-list
Re: new string-formatting preferred? (was What is this syntax ?)
On 06/20/2011 09:17 PM, Terry Reedy wrote:
On 6/20/2011 8:46 PM, Tim Chase wrote:
On 06/20/2011 05:19 PM, Ben Finney wrote:
“This method of string formatting is the new standard in
Python 3.0, and should be preferred to the % formatting
described in String Formatting Operations in new code.”
URL:http://docs.python.org/library/stdtypes.html#str.format
Is there a good link to a thread-archive on when/why/how .format(...)
became preferred to the % formatting?
That is a controversial statement.
I'm not sure whether you're controversial refers to
- the documentation at that link,
- Ben's quote of the documentation at that link,
- my quotation of Ben's quote of the documentation,
- or my request for a thread-archive on the when/why/how
I _suspect_ you mean the first one :)
I haven't seen any great wins of the new formatting over
the classic style.
It does not abuse the '%' operator,
Weighed against the inertia of existing
code/documentation/tutorials, I consider this a toss-up. If
.format() had been the preferred way since day#1, I'd grouse
about adding/overloading '%', but going the other direction,
there's such a large corpus of stuff using '%', the addition of
.format() feels a bit schizophrenic.
it does not make a special case of tuples (a source of bugs),
Having been stung occasionaly by this, I can see the benefit here
over writing the less-blatant
whatever %s % (tupleish,)
and it is more flexible, especially
indicating objects to be printed. Here is a simple example from my code
that would be a bit more difficult with %.
multi_warn = '''\
Warning: testing multiple {0}s against an iterator will only test
the first {0} unless the iterator is reiterable; most are not.'''.format
...
print(multiwarn('function'))
...
print(multiwarn('iterator'))
Does the gotcha of a non-restarting iterator trump pulling each
field you want and passing it explicitly? In pre-.format(), I'd
just use dictionary formatting:
we have %(food)s eggs and %(food)s, bacon eggs % {
food: spam, # or my_iterator.next()?
}
class chunk():
def __init__(self, a, b):
self.a,self.b = a,b
c=chunk(1, (3,'hi'))
print('{0.__class__.__name__} object has attributes int a{0.a}
and tuple b with members{0.b[0]} and{0.b[1]}'.format(c))
This was one of the new features I saw, and I'm not sure how I
feel about my strings knowing about my object structure. It
feels a bit like a violation of the old separation of content
and presentation. Letting string-formatting reach deeply into
objects makes it harder to swap out different object
implementations purely by analyzing the code. It also can put
onus on translators to know about your object model if your
format-strings come from an i18n source. I also see possible
mechanisms for malicious injection if the format-string comes
from an untrusted source (unlikely, but I've seen enough bad code
in production to make it at least imaginable).
The other new feature I saw was the use of __format__() which may
have good use-cases, but I don't yet have a good example of when
I'd want per-stringification formatting compared to just doing my
desired formatting in __str__() instead.
So even with your examples of differences, I don't get an
overwhelming feeling of wow, that *is* a much better way! like
I did with some of the other new features such as with or
changing print to a function.
Anyways, as you mention, I suspect blessing .format() as
preferred in the documentation was a bit contentious...with
enough code still running in 2.4 and 2.5 environments, it will be
a long time until I even have to think about it. I just wanted
to watch a replay of the decision-makers bashing it out :)
-tkc
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to iterate on a changing dictionary
Perhaps this is the simplest and best solution which appears in this case. Just
copy the desired items to a new dictionary and discard the original one.
import re
myDict={'a':'alpha','b':'beta','c':'charley','d':'disney'}
myNewDict={}
for k,v in myDict.iteritems():
if re.search(a,v)!=None:
myNewDict[k]=v
print myDict
print myNewDict
--
http://mail.python.org/mailman/listinfo/python-list
PSD to XHTML Conversion Services and PSD to HTML CSS Conversion Services, PSD to Joomla, Drupal, Wordpress Conversion
PSD to XHTML Conversion, PSD to HTML CSS, Joomla, Wordpress, Drupal, CMS, VBULLETIN, PHPBB and includes convert to XHTML like PSD to XHTML, web designing services, logos and banner design, website building, animation,presentations. for more details: http://www.xhtmlchamps.com -- http://mail.python.org/mailman/listinfo/python-list
Re: break in a module
On 21Jun2011 20:04, I wrote: | So the caller does an: | | import foo | | as normal, with no special wrapping. And the module goes: | | spam() | if condition: | raise StopIteration | ham() | cheese() Of course, that should be StopImport, not StopIteration. Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ Agree, for Law is costly. -- Very good advice to litigious Persons, founded upon Reason and Experience; for many Times the Charges of a Suit exceed the Value of the Thing in Dispute. - Bailey's dictionary, 1736 -- http://mail.python.org/mailman/listinfo/python-list
Web design services | website designing | hire a website designer | creative web design
web design services, XHTML Conversion Services, Offshore outsourcing web design SEO Expert Acedezines provides best services for your website design, search engine optimization, web design services, brochure design, flash intro animation, website designing, wordpress themes. for more details http://www.acedezines.com -- http://mail.python.org/mailman/listinfo/python-list
Emails backup in python 3.2
Hello, I'm looking for an idea how to backup emails retrieved by poplib and save them into mailbox.mbox file. The problem is the received message which is a list of bytes streams, mailbox.mbox don't expect a list. What conversion should I do? A file type io.StringIO ? decoding every bytes stream which might not have any declared codec? As far as python moved into unicode, why doesn't it handle these undecoded bytes as it was with strings before? -- goto /dev/null -- http://mail.python.org/mailman/listinfo/python-list
Don't understand SequenceMatcher from difflib
I have the following code I wrote.
==
from difflib import SequenceMatcher
import sys
write = sys.stdout.write
warn = sys.stderr.write
def program(argv):
ls1 = open(argv[1]).readlines()
ls2 = open(argv[2]).readlines()
matcher = SequenceMatcher(ls1, ls2)
s1 = 0
s2 = 0
print ls1
print ls2
warn(*** %d %d \n % (len(ls1), len(ls2)))
for e1, e2, lg in matcher.get_matching_blocks():
warn(*** %d %d %d\n % (e1, e2, lg))
for i in xrange(s1, e1):
write('- ')
write(ls1[i])
for i in xrange(s2, e2):
write('+ ')
write(ls2[i])
for i in xrange(e1, e1+lg):
write(' ')
write(ls1[i])
s1, s2 = e1 + lg, e2 + lg
if __name__ == '__main__':
program(sys.argv)
===
Now when I run it I get the following result:
python diff.py map.0 map.1
['\n', 'begin\n', ' a1\n', ' a2\n', ' a3\n', ' a4\n', ' a5\n', 'end\n',
'\n', 'begin\n', ' c1\n', ' c2\n', ' c3\n', ' c4\n', ' c5\n', ' c6\n', '
c7\n', 'end\n', '\n', 'begin\n', ' e1\n', ' e2\n', ' e3\n', ' e4\n', '
e5\n', ' e6\n', ' e7\n', ' e8\n', ' e9\n', 'end\n']
['\n', 'begin\n', ' a1\n', ' a2\n', ' a3\n', ' a4\n', ' a5\n', 'end\n',
'\n', 'begin\n', ' c1\n', ' c2\n', ' c3\n', ' c4\n', ' c5\n', ' c6\n', '
c7\n', 'end\n', '\n', 'begin\n', ' d1\n', ' d2\n', ' d3\n', 'end\n', '\n',
'begin\n', ' e1\n', ' e2\n', ' e3\n', ' e4\n', ' e5\n', ' e6\n', '
e7\n', ' e8\n', ' e9\n', 'end\n']
*** 30 36
*** 36 0 0
-
- begin
- a1
- a2
- a3
- a4
...
- Traceback (most recent call last):
File diff.py, line 31, in module
program(sys.argv)
File diff.py, line 21, in program
write(ls1[i])
IndexError: list index out of range
What I don't understand is: The first list is 30 items long and the second 36.
But the first match I get after calling get_matching_blocks says the match
starts
at item 36 of the first list.
Yes I noticed it is the sepcial last match with 0 siza,e but even if that would
be
correct because there would be no common items, the first number of the match
shouldn't be more than the length of the first list.
What am I doing wrong?
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to iterate on a changing dictionary
On 21/06/2011 12:51, Gurpreet Singh wrote:
Perhaps this is the simplest and best solution which appears in this case. Just
copy the desired items to a new dictionary and discard the original one.
import re
myDict={'a':'alpha','b':'beta','c':'charley','d':'disney'}
myNewDict={}
for k,v in myDict.iteritems():
if re.search(a,v)!=None:
myNewDict[k]=v
print myDict
print myNewDict
Using regex is overkill. Try this instead:
if a in v:
--
http://mail.python.org/mailman/listinfo/python-list
[ANN] Shed Skin 0.8
Hi all, I have just released version 0.8 of Shed Skin, an experimental (restricted-)Python-to-C++ compiler. Please see my blog for the full announcement: http://shed-skin.blogspot.com The Shed Skin homepage can be found here: http://shedskin.googlecode.com Thanks, Mark Dufour -- http://www.youtube.com/watch?v=E6LsfnBmdnk -- http://mail.python.org/mailman/listinfo/python-list
[OT] One click, one (buggy) life...
Hi everyone, I would like to share a youtube clip...one click costs nothing while can save lives sometimes (especially mine). http://www.youtube.com/watch?v=PiCeqtGHpJI Thanks a lot and cheers. DavCori http://www.ar4tro.com/welcome.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Rant on web browsers
This might help http://blog.stevenlevithan.com/archives/date-time-format -- http://mail.python.org/mailman/listinfo/python-list
Re: Is the mailing list to usenet gateway borked?
On 20 Jun 2011 23:49:11 GMT, Steven D'Aprano wrote: [snip] I will treat this as a bug in Pan, and take it to the appropriate forums, but for anyone who cares, here's one example: From: Steven D'Aprano steve+comp.lang.pyt...@pearwood.info Subject: Re: What is this syntax ? Newsgroups: comp.lang.python References: 4dfdfc99$0$715$426a3...@news.free.fr 4dfe10d1$0$28053 $426a3...@news.free.fr roy-2a6dc8.11394419062...@news.panix.com e724fc3e-8198-4fb7-b1d3-96834f3fa...@34g2000pru.googlegroups.com roy- dd0c35.16204819062...@news.panix.com mailman.164.1308517611.1164.python- l...@python.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: 19 Jun 2011 23:19:56 GMT Lines: 36 Message-ID: 4dfe841c$0$30002$c3e8da3$54964...@news.astraweb.com As you probably expected, that message is displayed normally by slrn. -- To email me, substitute nowhere-spamcop, invalid-net. -- http://mail.python.org/mailman/listinfo/python-list
Re: Do we still need to inherit from object to create new-style classes?
On 6/20/2011 9:26 PM, John Salerno wrote: I can't quite seem to find the answer to this anywhere. The book I'm reading right now was written for Python 3.1 and doesn't use (object), so I'm thinking that was just a way to force new-style classes in 2.x and is no longer necessary in 3.x. Is that right? (The documentation doesn't mention object anymore, Lib ref 2. builtin functions: object() Return a new featureless object. object is a base for all classes. It has the methods that are common to all instances of Python classes. This function does not accept any arguments. Note object does not have a __dict__, so you can’t assign arbitrary attributes to an instance of the object class. but elsewhere on the Python website it says the documentation hasn't been updated for new-style classes yet, hence my confusion.) What page? *It* may need updating ;-). The core 3.x docs have been updated by removing all reference to old-style classes and the modifier 'new-style'. The concept 'new-style' only exists in opposition to 'old-style'. 3.x just has classes, and all are subclasses of object. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
running an existing script
Hi, I'm trying to put together a lot of pieces of source code in matlab, java, perl and python. Im an expert when it comes to matlab, but novice in all the others listed above. However, I have integrated the java and perl code so they can be called from matlab. I know that there is a toolbox out there called Pymat but i think that uses 32bit activex so rules my configuration out. However I think I can hack in to the python command prompt from matlab. Basically I just want to run a single script from the python command window. Once I know how to do that I can be off on my way to perform the matlab interfacing. there is an example of the command I need in the python prompt at http://jboost.sourceforge.net/doc.html#cv . however, I can't seem to run the python script by typing the command on that link in the python prompt. Can I please ask how to set the current duirectory in python? the script I want to run is in a different directory to the one python is installed to -- http://mail.python.org/mailman/listinfo/python-list
Better way to iterate over indices?
I have always found that iterating over the indices of a list/tuple is not very clean: for i in range(len(myList)): doStuff(i, myList[i]) I know I could use enumerate: for i, v in enumerate(myList): doStuff(i, myList[i]) ...but that stiff seems clunky. Are there any better ways to iterate over the indices of a list /tuple? --Bill -- http://mail.python.org/mailman/listinfo/python-list
Re: parse date string having EDT
Thanks, My script should be platform independent, so I think filtering out time zone info is better. -- http://mail.python.org/mailman/listinfo/python-list
Re: Better way to iterate over indices?
On Tue, Jun 21, 2011 at 12:05 PM, Billy Mays no...@nohow.com wrote: I know I could use enumerate: for i, v in enumerate(myList): doStuff(i, myList[i]) ...but that stiff seems clunky. Why not: for i, v in enumerate(myList): doStuff(i, v) -- http://mail.python.org/mailman/listinfo/python-list
Re: running an existing script
Adam Chapman wrote:
Hi,
Howdy!
I'm trying to put together a lot of pieces of source code in matlab,
java, perl and python.
[snippety]
Basically I just want to run a single script from the python command
window. Once I know how to do that I can be off on my way to perform
the matlab interfacing.
there is an example of the command I need in the python prompt at
http://jboost.sourceforge.net/doc.html#cv .
That looks like a shell prompt, not a Python prompt
however, I can't seem to run the python script by typing the command
on that link in the python prompt.
Can I please ask how to set the current duirectory in python?
nfold.py is a python script -- you can't just type in the name once
inside python and have it work. It would require something like
-- import os
-- os.chdir('path/to/nfold.py') # don't include nfold.py ;)
-- import nfold
-- import sys
-- sys.argv = [--folds=5, --data=spambase.data,
... --spec=spambase.spec, --rounds=500, --tree=ADD_ALL,
... --generate ]
...
-- nfold.main() # assuming it has a main function that can be called
# in this manner
and that probably won't work. What you probably want to do is execute
the command python /path/to/nfold.py --fold=5 ... (include the
nfold.py this time ). I have no idea how to do that from Matlab.
Good luck!
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list
Re: Better way to iterate over indices?
On Tue, Jun 21, 2011 at 7:05 PM, Billy Mays no...@nohow.com wrote:
I have always found that iterating over the indices of a list/tuple is not
very clean:
for i in range(len(myList)):
doStuff(i, myList[i])
I know I could use enumerate:
for i, v in enumerate(myList):
doStuff(i, myList[i])
...but that stiff seems clunky.
You're not using it properly. Think about it. You're giving two names
- i and v. You've forgotten about v -
for i, v in enumerate('fish'):
... print i, v
...
0 f
1 i
2 s
3 h
HTH.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Better way to iterate over indices?
On Tue, Jun 21, 2011 at 11:05 AM, Billy Mays no...@nohow.com wrote: I have always found that iterating over the indices of a list/tuple is not very clean: for i in range(len(myList)): doStuff(i, myList[i]) I know I could use enumerate: for i, v in enumerate(myList): doStuff(i, myList[i]) ...but that stiff seems clunky. Why does enumerate seem clunky, other than the fact that you should probably have doStuff(i,v) instead of myList[i] in there? It's a bit more awkward than C's syntax, but since the typical use case is just iteration anyway, it's not a huge deal for those few cases where you actually need the indices. Are there any better ways to iterate over the indices of a list /tuple? --Bill -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Better way to iterate over indices?
Billy Mays wrote: I have always found that iterating over the indices of a list/tuple is not very clean: for i in range(len(myList)): doStuff(i, myList[i]) Definitely not beautiful. ;) I know I could use enumerate: for i, v in enumerate(myList): doStuff(i, myList[i]) If you actually need the index, then this is the way to do it. Note that in most cases, you don't need the index and can iterate directly: for v in myList: doStuff(v) From your sample code (assuming you don't need i) this does the same thing. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: running an existing script
On Tue, Jun 21, 2011 at 10:45 AM, Adam Chapman adamchapman1...@hotmail.co.uk wrote: Hi, I'm trying to put together a lot of pieces of source code in matlab, java, perl and python. Im an expert when it comes to matlab, but novice in all the others listed above. However, I have integrated the java and perl code so they can be called from matlab. I know that there is a toolbox out there called Pymat but i think that uses 32bit activex so rules my configuration out. However I think I can hack in to the python command prompt from matlab. Basically I just want to run a single script from the python command window. Once I know how to do that I can be off on my way to perform the matlab interfacing. there is an example of the command I need in the python prompt at http://jboost.sourceforge.net/doc.html#cv . however, I can't seem to run the python script by typing the command on that link in the python prompt. That command they show isn't run from a Python shell. It's run from either a Unix shell (bash and friends) or a Windows command prompt (cmd). If you want to run a script, you have to give the path to that script. ./ means the current directory and .. is the parent directory if you want to give relative paths, or you can just write out the whole file path. Can I please ask how to set the current duirectory in python? os.chdir changes the current directory, but you probably don't need to do that. the script I want to run is in a different directory to the one python is installed to -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
How to get return values of a forked process
Hello all,
I need some helped with forking. In my script, I fork a process. I
want to get return values from the child process.
This is the script that does the forking:
for x in (mylist):
pid = os.fork()
if pid:
pidList.append(pid)
else:
os.execv('/usr/bin/python',('/usr/bin/
python',myForkedScript))
for pid in pidList:
childPid, status = os.waitpid(pid,0)
# I think status should be the return value of the forked
process; I would expect status to be a 1 or a 0
myForkedScript has code like this:
if fail:
os._exit(1)
else:
os._exit(os.EX_OK)
Is using os._exit() the correct way to get a return value back to the
main process?
I thought the value 'n', passed in os._exit(n) would be the value I
get returned. In the case of a failure, I get 256 returned rather
than 1.
Thanks for the assistance!
IL
--
http://mail.python.org/mailman/listinfo/python-list
Simple question about loading a .glade UI file
Hi Everyone, So I'm tackling designing a non-CLI program using Glade. I went through some tutorials and it seems like I'm doing things right but I'm my UI won't load. Python keeps griping about could not create glade XML object. I have a .glade file called MainWindow.glade and my main window is called (predictably) winMain. Here is the code I'm using to load it: #!/usr/bin/env python import sys try: import pygtk pygtk.require(2.0) except: pass try: import gtk import gtk.glade except: print GTK could not be loaded. sys.exit(1) class GMB: def __init__(self): self.gladefile = MainWindow.glade self.wTree = gtk.glade.XML(self.gladefile) self.wTree.signal_autoconnect(self) self.window = self.wTree.get_widget(winMain) if(self.window): self.window.connect(destroy, gtk.main_quit) def on_winMain_delete(self, widget, dummy): gtk.main_quit() if __name__ == __main__: myGui = GMB() gtk.main() Is there any reason why I'd be getting this error from the code above? Both the UI file and the source code file are in the same directory. Thanks! Anthony -- http://mail.python.org/mailman/listinfo/python-list
Re: running an existing script
Adam Chapman wrote:
Thanks Ethan
No way could I have worked that out in my state of stress!
For your second idea, would I need to type that into the python command
line interface (the one that looks like a DOS window?
If you are actually in a python CLI, at the top of that screen does it
say something like
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit
(Intel)] on win32
Type help, copyright, credits or license for more information.
?
If yes, then what I wrote earlier should actually work (I downloaded
jBoost and looked at the nfold.py script). Here it is again:
-- import os
-- os.chdir('path/to/nfold.py') # don't include nfold.py ;)
-- import nfold
-- import sys
-- sys.argv = [nfold.py, --folds=5, --data=spambase.data,
... --spec=spambase.spec, --rounds=500, --tree=ADD_ALL,
... --generate ]
...
-- nfold.main()
I fixed the sys.argv line from last time.
Good luck!
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to get return values of a forked process
On Tue, Jun 21, 2011 at 12:26 PM, Ian ian.l...@rocketmail.com wrote: myForkedScript has code like this: if fail: os._exit(1) else: os._exit(os.EX_OK) Is using os._exit() the correct way to get a return value back to the main process? sys.exit() is the preferred way. I thought the value 'n', passed in os._exit(n) would be the value I get returned. In the case of a failure, I get 256 returned rather than 1. According to the docs, on Unix: Wait for completion of a child process, and return a tuple containing its pid and exit status indication: a 16-bit number, whose low byte is the signal number that killed the process, and whose high byte is the exit status (if the signal number is zero); the high bit of the low byte is set if a core file was produced. And on Windows: Wait for completion of a process given by process handle pid, and return a tuple containing pid, and its exit status shifted left by 8 bits (shifting makes cross-platform use of the function easier). (256 8) == 1 However, I would advise using the subprocess module for this instead of the os module (which is just low-level wrappers around system calls). -- http://mail.python.org/mailman/listinfo/python-list
Re: Don't understand SequenceMatcher from difflib
On 6/21/2011 9:43 AM, Antoon Pardon wrote:
matcher = SequenceMatcher(ls1, ls2)
...
What am I doing wrong?
Read the doc, in particular, the really stupid signature of the class:
class difflib.SequenceMatcher(isjunk=None, a='', b='', autojunk=True)
You are passing isjunk = ls1, a = ls2, and by default, b=''. So there
are no matches, len(a) = 36, len(b) = 0, and the dummy match is (36,0,0)
as you got.
There are also several example in the doc, all like
s = SequenceMatcher(None, abcd, abcd abcd) # or
s = SequenceMatcher(lambda x: x== , abcd, abcd abcd)
So you will get better results with
matcher = SequenceMatcher(None, ls1, ls2) # or
matcher = SequenceMatcher(a=ls1, b=ls2)
In the future, please try to simply examples before posting for help.
print(list(SequenceMatcher('a','abc').get_matching_blocks()))
shows the problem you posted in one easily read line of input and
output. I only waded through the distracting code and output you posted
to find that problem because I patched SequenceMatcher last fall.
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to get return values of a forked process
On Jun 21, 1:54 pm, Ian Kelly ian.g.ke...@gmail.com wrote: On Tue, Jun 21, 2011 at 12:26 PM, Ian ian.l...@rocketmail.com wrote: myForkedScript has code like this: if fail: os._exit(1) else: os._exit(os.EX_OK) Is using os._exit() the correct way to get a return value back to the main process? sys.exit() is the preferred way. I thought the value 'n', passed in os._exit(n) would be the value I get returned. In the case of a failure, I get 256 returned rather than 1. According to the docs, on Unix: Wait for completion of a child process, and return a tuple containing its pid and exit status indication: a 16-bit number, whose low byte is the signal number that killed the process, and whose high byte is the exit status (if the signal number is zero); the high bit of the low byte is set if a core file was produced. And on Windows: Wait for completion of a process given by process handle pid, and return a tuple containing pid, and its exit status shifted left by 8 bits (shifting makes cross-platform use of the function easier). (256 8) == 1 However, I would advise using the subprocess module for this instead of the os module (which is just low-level wrappers around system calls). Where did you find the Unix docs you pasted in? I didn't find it in the man pages. Thank you. Based on what you say, I will change my os._exit() to sys.exit(). -- http://mail.python.org/mailman/listinfo/python-list
Re: parse date string having EDT
You might consider trying dateutil.parser.parse (http://labix.org/python-dateutil#head-c0e81a473b647dfa787dc11e8c69557ec2c3ecd2) -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get return values of a forked process
One way is to use pipes, have a look at http://code.activestate.com/recipes/576709/ for example. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get return values of a forked process
Where did you find the Unix docs you pasted in? I didn't find it in the man pages. Thank you. Based on what you say, I will change my os._exit() to sys.exit(). http://docs.python.org/library/os.html#os.wait http://docs.python.org/library/os.html#os.waitpid I don't know what man pages you were looking at, but the Unix system call does work the same way; to extract the exit status in C you're supposed to use the WEXITSTATUS(status) macro, which is typically implemented as ((status 8) 0377). See: http://linux.die.net/man/2/waitpid -- http://mail.python.org/mailman/listinfo/python-list
How can I speed up a script that iterates over a large range (600 billion)?
I'm working on the Project Euler exercises and I'm stumped on problem 3: What is the largest prime factor of the number 600851475143 ? Now, I've actually written functions to get a list of the factors of any given number, and then another function to get the prime numbers from that list. It works fine with small numbers, but when I try to feed my get_factors function with the above number (600 billion), naturally it takes forever! But according to the Project Euler website: I've written my program but should it take days to get to the answer? Absolutely not! Each problem has been designed according to a one- minute rule, which means that although it may take several hours to design a successful algorithm with more difficult problems, an efficient implementation will allow a solution to be obtained on a modestly powered computer in less than one minute. But it definitely takes more than a minute, and I still haven't gotten it to end yet without just canceling it myself. Here is what I have so far. Initially the get_factors function just iterated over the entire range(2, n + 1), but since a number can't have a factor greater than half of itself, I tried to shorten the range by doing range(2, n //2), but that still leaves 300 billion numbers to go through. def get_factors(number): factors = [number] for n in range(2, number // 2): if number % n == 0: factors.append(n) return factors def get_primes(number_list): primes = number_list[:] for n in number_list: for x in range(2, n): if n % x == 0: primes.remove(n) break return primes print(max(get_primes(get_factors(600851475143 Also, I want to make it clear that I DO NOT WANT THE ANSWER. I really want to solve this myself, but the reason I'm asking about it is to see if there really is some way to change this code so that it can get an answer in less than one minute, as the website says should be possible. A hint about what I need to do would be nice, but not an answer. I just don't see any way to get the factors without iterating over the entire range of values, though. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I speed up a script that iterates over a large range (600 billion)?
On Tue, Jun 21, 2011 at 1:48 PM, John Salerno johnj...@gmail.com wrote: Here is what I have so far. Initially the get_factors function just iterated over the entire range(2, n + 1), but since a number can't have a factor greater than half of itself, I tried to shorten the range by doing range(2, n //2), but that still leaves 300 billion numbers to go through. Without giving you the answer, I will note that the range can be further reduced by quite a lot (I'm talking orders of magnitude, not just by half). def get_primes(number_list): primes = number_list[:] for n in number_list: for x in range(2, n): if n % x == 0: primes.remove(n) break return primes Also, primality testing and factorization are very similar problems, and the same range optimization could be applied here as well. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: basic bytecode to machine code compiler (part 3)
On 06/21/2011 06:55 AM, Ulrich Eckhardt wrote: Rouslan Korneychuk wrote: if i != pindex: (less if x= pivot else greater).append(x) Just curious, is there a reason why you wrote this last line that way instead of using a normal if/else clause? Cheers! Uli No special reason. I just like concise code. -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I speed up a script that iterates over a large range (600 billion)?
On 21-06-11 21:48, John Salerno wrote: I'm working on the Project Euler exercises and I'm stumped on problem 3: What is the largest prime factor of the number 600851475143 ? Now, I've actually written functions to get a list of the factors of any given number, and then another function to get the prime numbers from that list. It works fine with small numbers, but when I try to feed my get_factors function with the above number (600 billion), naturally it takes forever! You need a better algorithm to calculate primes, and iterate over primes instead of over the full (half, or even better, sqrt(n)) range of possible values. You also should optimize the stop condition, once you find that the number can no longer be divided by larger primes you can stop the loop. For instance to get the prime factors of the number 1000 you'd iterate over the prime numbers 2,3,5 and conclude that 1000=2*2*2*5*5*5, so 5 would be the largest prime factor. No need to try larger primes than 5, let alone go through 1..sqrt(1000). The Sieve of Eratosthenes is a well known algorithm to calculate primes with reasonable efficiency. Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get return values of a forked process
On Tue, Jun 21, 2011 at 12:26 PM, Ian ian.l...@rocketmail.com wrote: myForkedScript has code like this: if fail: os._exit(1) else: os._exit(os.EX_OK) Is using os._exit() the correct way to get a return value back to the main process? The correct way, no, but it is a correct way (and cheaper than using a pipe to pickle and unpickle failure, the way the subprocess module does it, for instance). In any case, you *should* call os._exit() either directly or indirectly after a successful fork but a failed exec. On Jun 21, 1:54 pm, Ian Kelly ian.g.ke...@gmail.com wrote: sys.exit() is the preferred way. Using sys.exit() after a fork() has other risks (principally, duplication of pending output when flushing write-mode streams), which is why os._exit() is provided. I thought the value 'n', passed in os._exit(n) would be the value I get returned. In the case of a failure, I get 256 returned rather than 1. According to the docs ... [snip documentation and description] However, I would advise using the subprocess module for this instead of the os module (which is just low-level wrappers around system calls). Indeed, subprocess gives you convenience, safety, and platform independence (at least across POSIX-and-Windows) with a relatively low cost. As long as the cost is low enough (and it probably is) I agree with this. In article d195a74d-e173-4168-8812-c03fc02e8...@fr19g2000vbb.googlegroups.com Ian ian.l...@rocketmail.com wrote: Where did you find the Unix docs you pasted in? I didn't find it in the man pages. Thank you. Based on what you say, I will change my os._exit() to sys.exit(). Not sure where Ian Kelly's documentation came from, but note that on Unix, the os module also provides os.WIFSIGNALED, os.WTERMSIG, os.WIFEXITED, and os.WEXITSTATUS for dissecting the status integer returned from the various os.wait* calls. Again, if you use the subprocess module, you are insulated from this sort of detail (which, as always, has both advantages and disadvantages). -- In-Real-Life: Chris Torek, Wind River Systems Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603 email: gmail (figure it out) http://web.torek.net/torek/index.html -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I speed up a script that iterates over a large range (600 billion)?
John Salerno wrote: I'm working on the Project Euler exercises and I'm stumped on problem 3: What is the largest prime factor of the number 600851475143 ? [ ... ] Here is what I have so far. Initially the get_factors function just iterated over the entire range(2, n + 1), but since a number can't have a factor greater than half of itself, I tried to shorten the range by doing range(2, n //2), but that still leaves 300 billion numbers to go through. def get_factors(number): factors = [number] for n in range(2, number // 2): if number % n == 0: factors.append(n) return factors def get_primes(number_list): primes = number_list[:] for n in number_list: for x in range(2, n): if n % x == 0: primes.remove(n) break return primes print(max(get_primes(get_factors(600851475143 Also, I want to make it clear that I DO NOT WANT THE ANSWER. I really want to solve this myself, but the reason I'm asking about it is to see if there really is some way to change this code so that it can get an answer in less than one minute, as the website says should be possible. A hint about what I need to do would be nice, but not an answer. I just don't see any way to get the factors without iterating over the entire range of values, though. It certainly can be done faster. I ran it against the factor finder that I wrote, and it popped up the answer mwilson@tecumseth:~$ bin/factors.py 600851475143 71 839 1471 ... before I could glance at my watch. factors.py works, as does yours, by testing for small factors first, but it divides them out as it goes, so it tends to do its work on smallish numbers. And since the smallest factors are taken out as soon as possible, they have to be the prime ones. Good hunting, Mel. -- http://mail.python.org/mailman/listinfo/python-list
sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
On 21-06-11 22:10, Irmen de Jong wrote: [stuff] I didn't read the last paragraph of John's message until just now, and now realize that what I wrote is likely way too much information for what he asked. I'm sorry. Next time I'll read everything until and including the last full stop. Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I speed up a script that iterates over a large range (600 billion)?
On 21/06/2011 20:48, John Salerno wrote: I'm working on the Project Euler exercises and I'm stumped on problem 3: What is the largest prime factor of the number 600851475143 ? Now, I've actually written functions to get a list of the factors of any given number, and then another function to get the prime numbers from that list. It works fine with small numbers, but when I try to feed my get_factors function with the above number (600 billion), naturally it takes forever! But according to the Project Euler website: I've written my program but should it take days to get to the answer? Absolutely not! Each problem has been designed according to a one- minute rule, which means that although it may take several hours to design a successful algorithm with more difficult problems, an efficient implementation will allow a solution to be obtained on a modestly powered computer in less than one minute. But it definitely takes more than a minute, and I still haven't gotten it to end yet without just canceling it myself. [snip] A non-prime is the product of a prime and another number which may or may not be a prime. Look for the smallest prime and repeat. On a modern PC, if it takes more than, say, a second for the given number, you're doing it wrong. :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: running an existing script
On Jun 21, 8:00 pm, Ethan Furman et...@stoneleaf.us wrote:
Adam Chapman wrote:
Thanks Ethan
No way could I have worked that out in my state of stress!
For your second idea, would I need to type that into the python command
line interface (the one that looks like a DOS window?
If you are actually in a python CLI, at the top of that screen does it
say something like
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit
(Intel)] on win32
Type help, copyright, credits or license for more information.
?
If yes, then what I wrote earlier should actually work (I downloaded
jBoost and looked at the nfold.py script). Here it is again:
-- import os
-- os.chdir('path/to/nfold.py') # don't include nfold.py ;)
-- import nfold
-- import sys
-- sys.argv = [nfold.py, --folds=5, --data=spambase.data,
... --spec=spambase.spec, --rounds=500, --tree=ADD_ALL,
... --generate ]
...
-- nfold.main()
I fixed the sys.argv line from last time.
Good luck!
~Ethan~
Thanks to both of you for your help.
It's getting late here, I'll give it another try tomorrow
--
http://mail.python.org/mailman/listinfo/python-list
Handling import errors
What is the pythonic way to handle imports error? What is bugging me is that the imports can't be inside a function (because I use them in different places in the script and thus they have to be in the global scope). I would write something like: try: -- http://mail.python.org/mailman/listinfo/python-list
Handling import errors
What is the pythonic way to handle imports error? What is bugging me
is that the imports can't be inside a function (because I use them in
different places in the script and thus they have to be in the global
scope). I would write something like:
try:
import foo
except ImportError:
logging.error('could not import foo')
sys.exit(1)
But logging is not configured at this point as my main() have not been
called yet.
Should I define a global variable and assign it to my module later? Or
should I let the exception happen and let the stack trace be the error
message?
--
http://mail.python.org/mailman/listinfo/python-list
Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
On Jun 21, 3:22 pm, Irmen de Jong ir...@-nospam-xs4all.nl wrote: On 21-06-11 22:10, Irmen de Jong wrote: [stuff] I didn't read the last paragraph of John's message until just now, and now realize that what I wrote is likely way too much information for what he asked. I'm sorry. Next time I'll read everything until and including the last full stop. Irmen Don't worry, I was still unclear about what to do after reading all the responses, even yours! But one thing that made me feel better was that I wasn't having a Python problem as much as a *math* problem. I changed my get_factors function to only go as far as the square root of the number in question, and that yielded an answer immediately. :) However, even after reading the Wikipedia page about prime numbers and trial division, I'm still a little unclear as to why the square root of the number is the upper bound of the range you need to check. -- http://mail.python.org/mailman/listinfo/python-list
Re: Handling import errors
* Guillaume Martel-Genest guillaum...@gmail.com [110621 12:53]:
What is the pythonic way to handle imports error? What is bugging me
is that the imports can't be inside a function (because I use them in
different places in the script and thus they have to be in the global
scope). I would write something like:
Suppose you do something like this:
try :
mod = __import__('mymodulename')
except ImportError:
pass ## replace with error handling here
so `mod' is global if you execute the above code as top-level,
*but* you can also pass `mod' as an argument like any other
variable, as far as I know and far as I have done.
--
Tim
tim at johnsons-web dot com or akwebsoft dot com
http://www.akwebsoft.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Handling import errors
try:
import foo
except ImportError:
logging.error('could not import foo')
sys.exit(1)
Why not just let the exception terminate the program? It will have even more
information about the problem that caused foo not to load.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Handling import errors
Guillaume Martel-Genest wrote: What is the pythonic way to handle imports error? What is bugging me is that the imports can't be inside a function (because I use them in different places in the script and thus they have to be in the global scope). Actually, you can if you declare them global: Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) [GCC 4.4.3] on linux2 Type help, copyright, credits or license for more information. def f(): ... global os ... import os ... dir (os) Traceback (most recent call last): File stdin, line 1, in module NameError: name 'os' is not defined f() dir (os) ['EX_CANTCREAT', 'EX_CONFIG', 'EX_DATAERR', 'EX_IOERR', 'EX_NOHOST', 'EX_NOINPUT', 'EX_NOPERM', 'EX_NOUSER', 'EX_OK', 'EX_OSERR', 'EX_OSFILE', 'EX_PROTOCOL', 'EX_SOFTWARE', 'EX_TEMPFAIL', 'EX_UNAVAILABLE', 'EX_USAGE', 'F_OK', 'NGROUPS_MAX', 'O_APPEND', 'O_ASYNC', 'O_CREAT', 'O_DIRECT', 'O_DIRECTORY', 'O_DSYNC', 'O_EXCL', 'O_LARGEFILE', 'O_NDELAY', 'O_NOATIME', 'O_NOCTTY', 'O_NOFOLLOW', 'O_NONBLOCK etc. Mel. -- http://mail.python.org/mailman/listinfo/python-list
Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
On 21-6-2011 23:09, John Salerno wrote: On Jun 21, 3:22 pm, Irmen de Jong ir...@-nospam-xs4all.nl wrote: On 21-06-11 22:10, Irmen de Jong wrote: [stuff] I didn't read the last paragraph of John's message until just now, and now realize that what I wrote is likely way too much information for what he asked. I'm sorry. Next time I'll read everything until and including the last full stop. Irmen Don't worry, I was still unclear about what to do after reading all the responses, even yours! But one thing that made me feel better was that I wasn't having a Python problem as much as a *math* problem. I changed my get_factors function to only go as far as the square root of the number in question, and that yielded an answer immediately. :) Ok, cool :) However, even after reading the Wikipedia page about prime numbers and trial division, I'm still a little unclear as to why the square root of the number is the upper bound of the range you need to check. If there is an exact divisor d = √n, then the quotient n/d is also a divisor of n, and that quotient is = √n. So if we don't find such a quotient before reaching √n, it's not useful to search for d √n (because no divisors would be found). Irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
On Tue, Jun 21, 2011 at 3:09 PM, John Salerno johnj...@gmail.com wrote: Don't worry, I was still unclear about what to do after reading all the responses, even yours! But one thing that made me feel better was that I wasn't having a Python problem as much as a *math* problem. I changed my get_factors function to only go as far as the square root of the number in question, and that yielded an answer immediately. :) However, even after reading the Wikipedia page about prime numbers and trial division, I'm still a little unclear as to why the square root of the number is the upper bound of the range you need to check. Careful, note that the greatest prime factor may actually be greater than the square root. It's just that it's possible to find it without iterating past the square root. This is because for each p that is a factor of n, q is also a factor of n, where p * q = n. If p sqrt(n), then q sqrt(n). Therefore you can find p by finding q and dividing n / q. -- http://mail.python.org/mailman/listinfo/python-list
Re: those darn exceptions
On Tue, 21 Jun 2011 01:43:39 +, Chris Torek wrote: But how can I know a priori that os.kill() could raise OverflowError in the first place? In article 4e006912$0$29982$c3e8da3$54964...@news.astraweb.com Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: You can't. Even if you studied the source code, you couldn't be sure that it won't change in the future. Or that somebody will monkey-patch os.kill, or a dependency, introducing a new exception. Indeed. However, if functions that know which exceptions they themselves can raise declare this (through an __exceptions__ attribute for instance), then whoever changes the source or monkey-patches os.kill can also make the appropriate change to os.kill.__exceptions__. More importantly though, most functions are reliant on their argument. You *cannot* tell what exceptions len(x) will raise, because that depends on what type(x).__len__ does -- and that could be anything. So, in principle, any function could raise any exception. Yes; this is exactly why you need a type-inference engine to make this work. In this case, len() is more (though not quite exactly) like the following user-defined function: def len2(x): try: fn = x.__len__ except AttributeError: raise TypeError(object of type %r has no len() % type(x)) return fn() eg: len(3) Traceback (most recent call last): File stdin, line 1, in module TypeError: object of type 'int' has no len() len2(3) Traceback (most recent call last): File stdin, line 1, in module File stdin, line 5, in len2 TypeError: object of type type 'int' has no len() In this case, len would not have any __exceptions__ field (or if it does, it would not be a one-element tuple, but I currently think it makes more sense for many of the built-ins to resort to rules in the inference engine). This is also the case for most operators, e.g., ordinary + (or operator.add) is syntactic sugar for: first_operand.__add__(second_operand) or: second_operand.__radd__(first_operand) depending on both operands' types and the first operand's __add__. The general case is clearly unsolveable (being isomorphic to the halting problem), but this is not in itself an excuse for attempting to solve more-specific cases. A better excuse -- which may well be better enough :-) -- occurs when the specific cases that *can* be solved are so relatively-rare that the approach degenerates into uselessness. It is worth noting that the approach I have in mind does not survive pickling, which means a very large subset of Python code is indigestible to a pylint-like exception-inference engine. Another question -- is the list of exceptions part of the function's official API? *All* of the exceptions listed, or only some of them? All the ones directly-raised. What to do about invisible dependencies (such as those in len2() if len2 is invisible, e.g., coded in C rather than Python) is ... less obvious. :-) In general, you can't do this at compile-time, only at runtime. There's no point inspecting len.__exceptions__ at compile-time if len is a different function at runtime. Right. Which is why pylint is fallible ... yet pylint is still valuable. At least, I find it so. It misses a lot of important things -- it loses types across list operations, for instance -- but it catches enough to help. Here is a made-up example based on actual errors I have found via pylint: doc class Frob(object): doc def __init__(self, arg1, arg2): self.arg1 = arg1 self.arg2 = arg2 def frob(self, nicate): frobnicate the frob self.arg1 += nicate def quux(self): return the frobnicated value example = self # demonstrate that pylint is not using the *name* return example.argl # typo, meant arg1 ... $ pylint frob.py * Module frob E1101: 15:Frob.quux: Instance of 'Frob' has no 'argl' member (Loses types across list operations means that, e.g.: def quux(self): return [self][0].argl hides the type, and hence the typo, from pylint. At some point I intend to go in and modify it to track the element-types of list elements: in enough cases, a list's elements all have the same type, which means we can predict the type of list[i]. If a list contains mixed types, of course, we have to fall back to the failure-to-infer case.) (This also shows that much real code might raise IndexError: any list subscript that is out of range does so. So a lot of real functions *might* raise IndexError, etc., which is another argument that in real code, an exception inference engine will wind up concluding that every line might raise every exception. Which might be true, but I still believe, for the moment, that a tool for inferring exceptions would have some value.) -- In-Real-Life: Chris Torek, Wind River
Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
On Jun 21, 4:41 pm, Ian Kelly ian.g.ke...@gmail.com wrote: On Tue, Jun 21, 2011 at 3:09 PM, John Salerno johnj...@gmail.com wrote: Don't worry, I was still unclear about what to do after reading all the responses, even yours! But one thing that made me feel better was that I wasn't having a Python problem as much as a *math* problem. I changed my get_factors function to only go as far as the square root of the number in question, and that yielded an answer immediately. :) However, even after reading the Wikipedia page about prime numbers and trial division, I'm still a little unclear as to why the square root of the number is the upper bound of the range you need to check. Careful, note that the greatest prime factor may actually be greater than the square root. It's just that it's possible to find it without iterating past the square root. This is because for each p that is a factor of n, q is also a factor of n, where p * q = n. If p sqrt(n), then q sqrt(n). Therefore you can find p by finding q and dividing n / q. Oh! Now it makes sense! That first sentence helped to put it into perspective, too. The Wikipedia page says more or less the same thing, but this paragraph just made more sense to me. :) Thanks for the all the advice everyone. Now I'm on to problem #4, and I'm stumped again, but that's what's fun! :) -- http://mail.python.org/mailman/listinfo/python-list
Re: new string-formatting preferred? (was What is this syntax ?)
On 6/21/2011 7:33 AM, Tim Chase wrote:
On 06/20/2011 09:17 PM, Terry Reedy wrote:
On 6/20/2011 8:46 PM, Tim Chase wrote:
On 06/20/2011 05:19 PM, Ben Finney wrote:
“This method of string formatting is the new standard in
Python 3.0, and should be preferred to the % formatting
described in String Formatting Operations in new code.”
URL:http://docs.python.org/library/stdtypes.html#str.format
Is there a good link to a thread-archive on when/why/how .format(...)
became preferred to the % formatting?
That is a controversial statement.
I'm not sure whether you're controversial refers to
- the documentation at that link,
- Ben's quote of the documentation at that link,
- my quotation of Ben's quote of the documentation,
- or my request for a thread-archive on the when/why/how
I _suspect_ you mean the first one :)
I meant the preceding statement (derived from the linked source, but
that is not important) that .format is preferred to %. Guido prefers it.
I prefer it. At least a couple of developers vocally do not prefer it
and might prefer that the statement was not there. Guido recognizes that
deprecation of % formatting would at least require a conversion function
that does not now exist.
I see that the linked doc says 'in new code'. That makes the statement
less (but only less) controversial.
I haven't seen any great wins of the new formatting over
the classic style.
It does not abuse the '%' operator,
Weighed against the inertia of existing code/documentation/tutorials, I
consider this a toss-up. If .format() had been the preferred way since
day#1, I'd grouse about adding/overloading '%', but going the other
direction, there's such a large corpus of stuff using '%', the addition
of .format() feels a bit schizophrenic.
it does not make a special case of tuples (a source of bugs),
Having been stung occasionaly by this, I can see the benefit here over
writing the less-blatant
whatever %s % (tupleish,)
and it is more flexible, especially
indicating objects to be printed. Here is a simple example from my code
that would be a bit more difficult with %.
multi_warn = '''\
Warning: testing multiple {0}s against an iterator will only test
the first {0} unless the iterator is reiterable; most are not.'''.format
...
print(multiwarn('function'))
...
print(multiwarn('iterator'))
Does the gotcha of a non-restarting iterator
Huh? What iterator?
trump pulling each field you want and passing it explicitly?
Huh? I explicitly pass the strings to be printed.
In pre-.format(), I'd just use dictionary formatting:
we have %(food)s eggs and %(food)s, bacon eggs % {
food: spam, # or my_iterator.next()?
}
A better parallel to my example would be
menu = We have %(meat)s eggs or %(meat)s and potatoes.
print(menu % {'meat':'spam'})
print(menu % {'meat':'ham'})
The exact duplicate of that with .format is
menu = We have {meat} eggs or {meat} and potatoes..format
print(menu(meat = 'spam'))
print(menu(meat = 'ham'))
One knock against '.format' is that it is 6 chars more that '%'. But for
repeat usage, it is only needed once. And look: '%(meat)s' is 2 more
chars than '{meat}' and, to me, {} is easier to type than (). Then %
{'meat':spam} is 3 more chars than (meat = 'ham') and definitely
harder to type. While I prefer '}' to ')', I prefer '))' to the mixed
'})'. The % way is at least 'a bit more difficult' even compared to the
longer and harder .format with named fields.
menu = We have {0} eggs or {0} and potatoes..format
print(menu('spam'))
print(menu('ham'))
it a little easier yet, though perhaps less clear, especially if there
were multiple substitutions.
The other new feature I saw was the use of __format__() which may have
good use-cases, but I don't yet have a good example of when I'd want
per-stringification formatting compared to just doing my desired
formatting in __str__() instead.
__str__ always returns the same string for an instance in a given state.
Similarly, __float__ and __int__ will return the same float or int
version of an unchanged instance. __format__(spec) can directly adjust
the result according to spec without the restriction of going through an
intermediary str, int, or float.
Suppose one had a Money class with currency and decimal amount fields.
.__str__ can add a currency symbol (before or after as appropriate) but
has to use a standard format for the amount field. .__float__ can be
post-processed according to a %...f spec, but cannot include a currency
symbol. Money.__format__(self,spec) can format the amount at it wishes,
including its rounding rules, *and* add a currency symbol.
Or suppose one has a multi-precision float. %80.40f will require an mpf
instance to appoximate itself as a float, possibly with error.
mpg.__format__ should be able to do better.
(Sadly, this new ability to more accurately represent objects is not yet
used for ints and is broken for fractions.Fraction. I will probably post
issues on the
Re: something about performence
On 6/20/2011 10:59 PM, king6c...@gmail.com wrote:
Hi，
I have two large files,each has more than 2 lines,and each line
consists of two fields,one is the id and the other a value,
the ids are sorted.
for example:
file1
(uin_a y)
1 1245
2 12333
3 324543
5 3464565
file2
(uin_b gift)
1 34545
3 6436466
4 35345646
5 463626
I want to merge them and get a file,the lines of which consists of an id
and the sum of the two values in file1 and file2。
the codes are as below:
One minor thing you can do is use bound methods
uin_y=open('file1')
uin_gift=open(file2')
ynext = open('file1').next
gnext = open(file1').next
y_line=uin_y.next()
gift_line=uin_gift.next()
y_line = ynext()
gift_list = gnext()
and similarly for all .next appearances in what follows.
while 1:
try:
uin_a,y=[int(i) for i in y_line.split()]
This creates an unnecessary second temporary list. Unroll the loop.
pair = y_line.split
uin_a = int(pair[0])
y = int(pair[1])
uin_b,gift=[int(i) for i in gift_line.split()]
same for this line
if uin_a==uin_b:
score=y+gift
print uin_a,score
y_line=uin_y.next()
gift_line=uin_gift.next()
if uin_auin_b:
print uin_a,y
y_line=uin_y.next()
if uin_auin_b:
print uin_b,gift
gift_line=uin_gift.next()
except StopIteration:
break
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Finding greatest prime factor, was Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
On Wed, Jun 22, 2011 at 7:48 AM, John Salerno johnj...@gmail.com wrote: Thanks for the all the advice everyone. Now I'm on to problem #4, and I'm stumped again, but that's what's fun! :) So now that you've solved it, I'd like to see some fast one-liners to do the job. (Since Python cares about whitespace, it might not _technically_ fit on one line, but in the spirit of one-liners, try to keep it short.) Here's what I did in Pike, very quickly: exec 600851475143; for (int i=2;iret;++i) while (ret%i==0) ret/=i Porting it to Python: ret,i=600851475143,2 while iret: while not ret%i: ret//=i Bring on your simpler solutions! ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: new string-formatting preferred? (was What is this syntax ?)
On 06/21/2011 05:19 PM, Terry Reedy wrote:
On 6/21/2011 7:33 AM, Tim Chase wrote:
URL:http://docs.python.org/library/stdtypes.html#str.format
Is there a good link to a thread-archive on when/why/how .format(...)
became preferred to the % formatting?
That is a controversial statement.
I'm not sure whether you're controversial refers to
- the documentation at that link,
I meant the preceding statement (derived from the linked source, but
that is not important) that .format is preferred to %.
I guess then with all the contention, having such a vocal
preference in the docs (even if tempered by in new code) seems
to unnecessarily polarize when I find myself very meh in either
direction.
multi_warn = '''\
Warning: testing multiple {0}s against an iterator will only test
the first {0} unless the iterator is reiterable; most are not.'''.format
...
print(multiwarn('function'))
...
print(multiwarn('iterator'))
Does the gotcha of a non-restarting iterator
Huh? What iterator?
Your string-body text warns about behavior regarding iterators.
I was thrown by your warning.
The other new feature I saw was the use of __format__()
Suppose one had a Money class with currency and decimal amount fields.
.__str__ can add a currency symbol (before or after as appropriate) but
has to use a standard format for the amount field. .__float__ can be
post-processed according to a %...f spec, but cannot include a currency
symbol. Money.__format__(self,spec) can format the amount at it wishes,
including its rounding rules, *and* add a currency symbol.
A Money class was one of the first things I thought of, but
figured that would be better relegated to an i18n wrapper if you
wanted it. Such a wrapper would handle currency-symbol choice
positioning positioning (before vs. after; relation to the +/-;
optional characters for thousands-separators and decimal
separators; and partitioning at thousands-or-other-multiples, etc).
Or suppose one has a multi-precision float. %80.40f will require an mpf
instance to appoximate itself as a float, possibly with error.
mpg.__format__ should be able to do better.
This case makes a better argument, showing me some new value
added by __format__().
-tkc
--
http://mail.python.org/mailman/listinfo/python-list
Re: Finding greatest prime factor, was Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
Oops, realized after posting that there's a bug in my code - it returns 1 for a perfect square. Need another check in the 'while' loop, thus: On Wed, Jun 22, 2011 at 8:59 AM, Chris Angelico ros...@gmail.com wrote: exec 600851475143; for (int i=2;iret;++i) while (ret%i==0 reti) ret/=i while not ret%i and reti: Definitely room for improvement here! ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I speed up a script that iterates over a large range (600 billion)?
On 6/21/2011 3:48 PM, John Salerno wrote: Absolutely not! Each problem has been designed according to a one- minute rule, which means that although it may take several hours to design a successful algorithm with more difficult problems, an efficient implementation will allow a solution to be obtained on a modestly powered computer in less than one minute. That statement is for C, not Python. Python is efficient with human time, but not machine time. If something really takes a minute in C, allow yourself at least 10 minutes or even more with plain CPython. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Using django ORM from web browser and from command line apps
Hi, I'm having a django browser application. There's certain administrative tasks, that I'd like to perform from the command line (cronjob or manually). As these scripts might be huge and might consume quite some memory I'd prefer, that they were not part of the normal application and would just consume memory during the administrative phase. I am not sure, whether this is a use case really being intended. I wanted to know whether there are any precautions to take if I do this. The one issue, that I could imagine is that Django (if I understood correctly) performs normally database transaction based an a http request. What would happen if I don't do anythong special in a script. Would the entire runtime of the script be considered a transaction? Are there any special commands that I have to use in order to indicate when a transaction starts / stops? Thanks in advance for suggestions. Please look also at a related question. -- http://mail.python.org/mailman/listinfo/python-list
Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
2011/6/21 John Salerno johnj...@gmail.com: However, even after reading the Wikipedia page about prime numbers and trial division, I'm still a little unclear as to why the square root of the number is the upper bound of the range you need to check. -- There are likely be some more elaborated proofs, but it seems sufficiently evident, that with the factors being the square root you get some kind of middle position; with other factors (e.g. two for simplicity), one of them could be greater, while the another has to be smaller; this smaller one would have been discovered already, while searching continuously until the square root of the given number. vbr -- http://mail.python.org/mailman/listinfo/python-list
using only the django ORM (DB access model) and nothing else.
Hi, I have a small application running on a host without web server and without any need for django except its ORM accessing data bases without explicitely writing sql queries.) I assume there's many libraries (SQL Alchemy or others), which could do this job. and which migh have less overhead than django. As I am already implementing a web server application with django on another host I wanted to use the same syntax / API for my non web application. Now my question: What would be the minimal setup required to use django orms and nothing else. What entries could I remove from settings.py would I still have to add INSATLLED_APPS to the settings or could I just write one script defining only defining settings.DATABASES, and the typical contents of models.py of an application. Thanks in advance for some suggestions or ideas how you would approach writing a tiny non web application with django.db.models.Models -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I speed up a script that iterates over a large range (600 billion)?
Terry Reedy tjre...@udel.edu writes: efficient implementation will allow a solution to be obtained on a modestly powered computer in less than one minute. If something really takes a minute in C, allow yourself at least 10 minutes or even more with plain CPython. No. The idea of the Euler problems is to think up sane algorithms to solve them, not micro-optimize or use low level languages on crappy algorithms. n=600851475143 for d in xrange(2,n): if d*d n: break while n%d == 0: n //= d print n finishes on my laptop with no noticable pause. The trick is to stop testing once you hit the square root of the number. There is at least one extremely obvious optimization I didn't bother with (above 2, only test odd divisors), that would have doubled the speed on top of that. -- http://mail.python.org/mailman/listinfo/python-list
Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
John Salerno johnj...@gmail.com writes: However, even after reading the Wikipedia page about prime numbers and trial division, I'm still a little unclear as to why the square root of the number is the upper bound of the range you need to check. Suppose p is the smallest divisor of n, and p sqrt(n). (Divisor here means p=1 doesn't count). p being a divisor of n means there is some q such that n = pq. That means q = n / p. If p sqrt(n) that means that q must be sqrt(n). But that contradicts the claim that p is the smallest divisor. So we know that if there is a divisor at all, it must be = sqrt(n) and if we don't find one by then, n must be prime. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using django ORM from web browser and from command line apps
On Tue, Jun 21, 2011 at 5:39 PM, News123 news1...@free.fr wrote: Hi, I'm having a django browser application. There's certain administrative tasks, that I'd like to perform from the command line (cronjob or manually). As these scripts might be huge and might consume quite some memory I'd prefer, that they were not part of the normal application and would just consume memory during the administrative phase. I am not sure, whether this is a use case really being intended. It sounds like you probably want a custom management command: https://docs.djangoproject.com/en/1.3/howto/custom-management-commands/ I wanted to know whether there are any precautions to take if I do this. The one issue, that I could imagine is that Django (if I understood correctly) performs normally database transaction based an a http request. If you have the TransactionMiddleware enabled, yes. Otherwise the default is to commit everything immediately. What would happen if I don't do anythong special in a script. Would the entire runtime of the script be considered a transaction? The default here is also to commit everything immediately. Since there is no HTTP request, the TransactionMiddleware does not get invoked even if enabled. For controlled transactions you will need to use the commit_on_success or commit_manually decorator / context-managers: https://docs.djangoproject.com/en/1.3/topics/db/transactions/ Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Tkinter/scrollbar/canvas question
Dear Pythonistas, I've been trying to write a script that results in a set of widgets in a scrollable widget. Since tkinter is the gui for Python (and about the only one that I can be sure will not require additional software installation), I went that route, and wrote the following: from tkinter import * class ShowList(Frame): def __init__(self, root): Frame.__init__(self, root) self.grid() self.draw_widgets() def draw_widgets(self): cframe = Frame() cframe.grid(row=1, sticky=N+S+E+W) canv = Canvas(cframe) canv[scrollregion]=canv.bbox(ALL) canv.grid(row=0, column=0, sticky=N+S+E+W) vscroll = Scrollbar(cframe, orient=VERTICAL, command=canv.yview) hscroll = Scrollbar(cframe, orient=HORIZONTAL, command=canv.xview) vscroll.grid(row=0, column=1, sticky=N+S) hscroll.grid(row=1, column=0, sticky=E+W) canv[xscrollcommand] = hscroll.set canv[yscrollcommand] = vscroll.set aframe = Frame(canv) canv.create_window(0,0,window=aframe, anchor=N+W) for i in range(0,100): Label(aframe, text=str(i), anchor=N+W).grid(row=i, column=0) root = Tk() m=ShowList(root) root.mainloop() ...which is great in that it does display the list of items (in this case, a bunch of Labels) but not so great in the fact that the vertical scrollbar's scroll thumb is the list of the entire scroll trough, making skip down/up a screenful impossible. Both the horizontal scrollbar and the vertical scrollbar will scroll until the list is off the screen (not the desired behavior). Initially, I thought the canv[scrollregion] = canv.bbox(ALL) would take care of the off-screen scrolling by making the scrollable region no larger than it needs to be. Reading many examples of Python+tkinter code, intended to demonstrate the way to code scrollbars and canvases, didn't enlighten me enough to figure out the bug(s) in my code. Any suggestions as to what I did wrong? Many thanks in advance, --agb -- http://gall.mine.nu free to get comics,free chat-1,dc++ (dcplusplus), mute webcache,group update program,torrent,atomic time server,tool to find your ip -- http://mail.python.org/mailman/listinfo/python-list
Re: Using django ORM from web browser and from command line apps
Hi Ian, On 06/22/2011 02:09 AM, Ian Kelly wrote: On Tue, Jun 21, 2011 at 5:39 PM, News123 news1...@free.fr wrote: I'm having a django browser application. There's certain administrative tasks, that I'd like to perform from the command line (cronjob or manually). It sounds like you probably want a custom management command: https://docs.djangoproject.com/en/1.3/howto/custom-management-commands/ Didn't know this existed. Yes this is definitely a way to go. At least for part of the commands, that I plan to implement. Out of curiousity: Do you know whether the imports would be executed for each potential command as soon as I call manage.py or only 'on demand'? I wanted to know whether there are any precautions to take if I do this. The one issue, that I could imagine is that Django (if I understood correctly) performs normally database transaction based an a http request. If you have the TransactionMiddleware enabled, yes. Otherwise the default is to commit everything immediately. Good to know. What would happen if I don't do anythong special in a script. Would the entire runtime of the script be considered a transaction? The default here is also to commit everything immediately. Since there is no HTTP request, the TransactionMiddleware does not get invoked even if enabled. For controlled transactions you will need to use the commit_on_success or commit_manually decorator / context-managers: https://docs.djangoproject.com/en/1.3/topics/db/transactions/ Thanks this url is very clear. :-) I will use the commit model as required for my different tasks -- http://mail.python.org/mailman/listinfo/python-list
Re: using only the django ORM (DB access model) and nothing else.
On 06/22/2011 01:51 AM, News123 wrote:
Hi,
I have a small application running on a host without web server and
without any need for django except its ORM accessing data bases without
explicitely writing sql queries.)
I assume there's many libraries (SQL Alchemy or others), which could do
this job. and which migh have less overhead than django.
As I am already implementing a web server application with django on
another host I wanted to use the same syntax / API for my non web
application.
Now my question:
What would be the minimal setup required to use django orms and nothing
else.
What entries could I remove from settings.py
would I still have to add INSATLLED_APPS to the settings or could I just
write one script
defining only defining settings.DATABASES, and the typical contents of
models.py of an application.
Thanks in advance for some suggestions or ideas how you would approach
writing a tiny non web application with django.db.models.Models
I made a very first brute force test:
settings.py: (only DATABASES is set)
===
import os
MYDIR = os.path.abspath(os.path.dirname(__file__))
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(MYDIR, tiny.db),
'HOST': '',
'USER': '',
'PASSWORD': '',
'PORT': '',
}
}
myapp.py
==
#!/usr/bin/env python
import os
# just set the env prior to importing a django module
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
from django.db import models
print yes this line is executed
# this will fail :-(
class Mini(models.Model):
name = models.CharField(max_length=80)
###
If running myapp.py I get following output:
yes this line is executed
Traceback (most recent call last):
File ./myapp.py, line 11, in module
class Mini(models.Model):
File
/opt/my_python/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/db/models/base.py,
line 52, in __new__
kwargs = {app_label: model_module.__name__.split('.')[-2]}
IndexError: list index out of range
(my_python)n1234@mypc:~/minidjango$
So I need at least a little more to make my script work.
--
http://mail.python.org/mailman/listinfo/python-list
Security test of embedded Python
I'm involved in the construction of an environment that allows end users to supply scripts which will then run on our servers. We need to be able to offer the full flexibility of a scripting language, but without the risk of compromise to our computers. To that end, we have set up a system with pretty much the same facilities as our live system will have, and are offering this to the world to hammer on - and requesting the world's assistance in hunting down bugs. The environment is Python 3.3a0 embedded in C++, running on Linux. It's currently home-hosted to keep things simple, with only one port forwarded to it from our NAT router (so don't bother port scanning, you aren't looking at Monty). And yes, that's right. I have no imagination when it comes to names. Our test box really is called Monty. And to sign up for our forums, you'll need to prove you're a human by knowing that the name Python goes with Monty. Launch page: http://www.pythontest.com/ PHPBB forum: http://www.pythontest.com/forum/ (feedback here please, no need to clutter the python-list) Actual thing to whump into submission: http://www.pythontest.com:8000/ Find a bug, get noted as a contributor! :) Thanks! Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Is the mailing list to usenet gateway borked?
Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes: The last couple of messages on this list python-list@python.org show up fine on the mailman archives, but are empty posts on comp.lang.python. Is there a problem with the mail - usenet gateway? I don't see empty messages through Usenet. However, at around the time you reported this, I started seeing some (not many) messages each day with strange mangling of the header: the Date field showing Unix epoch, the Subject field truncated, and the From field showing the remainder from the Subject. Another distortion is that the summary of message size (which I think is derived from the Lines field) shows zero for all those problematic messages. That may indicate a link between the problems we're seeing. For example: = O [20110621T054352: 2.0k: deathweaselx86 ]─── Is there any advantage or disadvantage to using sets over list comps to ensure a list O [20110621T111058: 2.8k: Steven D'Aprano ]├─► [...] O [20110621T135956: 1.9k: rusi]└─► [...] O [19700101T10: 0.0k: comps to ensure a li]*── Re: Is there any advantage or disadvantage to using sets over list O 19700101T10: 0.0k: comps to ensure a li └─► [...] O [20110618T161430: 3.5k: Gregory Ewing ]─── ANN: PyGUI 2.5.1 = The problematic messages in the above example are Message-ID: mailman.191.1308600028.1164.python-l...@python.org and mailman.213.1308627006.1164.python-l...@python.org. The messages, when retrieved, have the right content and even the fields in the header appear fine. My news service for ‘comp.lang.python’ is currently provided from ‘astraweb.com’. -- \ “I'm a born-again atheist.” —Gore Vidal | `\ | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: using only the django ORM (DB access model) and nothing else.
On Tue, Jun 21, 2011 at 6:42 PM, News123 news1...@free.fr wrote:
###
If running myapp.py I get following output:
yes this line is executed
Traceback (most recent call last):
File ./myapp.py, line 11, in module
class Mini(models.Model):
File
/opt/my_python/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/db/models/base.py,
line 52, in __new__
kwargs = {app_label: model_module.__name__.split('.')[-2]}
IndexError: list index out of range
(my_python)n1234@mypc:~/minidjango$
So I need at least a little more to make my script work.
There's a bit of magic in the way Django finds things, and I think
you'll still need to keep the basic structure of a Django project --
models should be in a models.py file located in an app package,
which should be included in the INSTALLED_APPS setting. You just
won't have any views or urlconfs or templates or admin sites or
anything like that.
--
http://mail.python.org/mailman/listinfo/python-list
Re: using only the django ORM (DB access model) and nothing else.
In article 4e012e8d$0$23682$426a3...@news.free.fr, News123 news1...@free.fr wrote: Hi, I have a small application running on a host without web server and without any need for django except its ORM accessing data bases without explicitely writing sql queries.) You would do much better to ask this question on the django mailing list (http://groups.google.com/group/django-users). I assume there's many libraries (SQL Alchemy or others), which could do this job. and which migh have less overhead than django. Ugh. I've played with SQL Alchemy a few times and every time I've run away screaming in the other direction. I can see how it's useful if you need to be totally cross-platform, but, man, if that's what it takes to be cross platform, I'm happy being a MySQL bigot all day long. As I am already implementing a web server application with django on another host I wanted to use the same syntax / API for my non web application. Now my question: What would be the minimal setup required to use django orms and nothing else. I don't see any reason you couldn't use the Model layer by itself, if you want to. It pretty much stands on its own. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using django ORM from web browser and from command line apps
On Tue, Jun 21, 2011 at 6:21 PM, News123 news1...@free.fr wrote: Out of curiousity: Do you know whether the imports would be executed for each potential command as soon as I call manage.py or only 'on demand'? Off the top of my head, I don't know. -- http://mail.python.org/mailman/listinfo/python-list
Re: using only the django ORM (DB access model) and nothing else.
It seems I found a solution (refer to end of this tessage).
Not sure though if there are any drawbacks or if this method of working
could cause any other issues.
On 06/22/2011 02:42 AM, News123 wrote:
On 06/22/2011 01:51 AM, News123 wrote:
Hi,
I have a small application running on a host without web server and
without any need for django except its ORM accessing data bases without
explicitely writing sql queries.)
I assume there's many libraries (SQL Alchemy or others), which could do
this job. and which migh have less overhead than django.
As I am already implementing a web server application with django on
another host I wanted to use the same syntax / API for my non web
application.
Now my question:
What would be the minimal setup required to use django orms and nothing
else.
What entries could I remove from settings.py
would I still have to add INSATLLED_APPS to the settings or could I just
write one script
defining only defining settings.DATABASES, and the typical contents of
models.py of an application.
Thanks in advance for some suggestions or ideas how you would approach
writing a tiny non web application with django.db.models.Models
I made a very first brute force test:
settings.py: (only DATABASES is set)
===
import os
MYDIR = os.path.abspath(os.path.dirname(__file__))
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(MYDIR, tiny.db),
'HOST': '',
'USER': '',
'PASSWORD': '',
'PORT': '',
}
}
myapp.py
==
#!/usr/bin/env python
import os
# just set the env prior to importing a django module
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
from django.db import models
print yes this line is executed
# this will fail :-(
class Mini(models.Model):
name = models.CharField(max_length=80)
###
If running myapp.py I get following output:
yes this line is executed
Traceback (most recent call last):
File ./myapp.py, line 11, in module
class Mini(models.Model):
File
/opt/my_python/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/db/models/base.py,
line 52, in __new__
kwargs = {app_label: model_module.__name__.split('.')[-2]}
IndexError: list index out of range
(my_python)n1234@mypc:~/minidjango$
So I need at least a little more to make my script work.
directory structure is now
settings.py
myapp/__init__.py
myapp/models.py
tst.py
settings.py
import os
MYDIR = os.path.abspath(os.path.dirname(__file__))
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(MYDIR, tiny.db),
'HOST': '',
'USER': '',
'PASSWORD': '',
'PORT': '',
}
}
INSTALLED_APPS = (
'myapp',
)
myapp/models.py
from django.db import models
class Mini(models.Model):
name = models.CharField(max_length=80)
tst.py
---
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
import myapp.models as mymodels
for name in [one, two, three]:
mymodels.Mini(name=name).save()
print mymodels.Mini.objects.all().values()
now I can call syncdb with:
django-admin syncdb --settings=settings --pythonpath=`pwd`
and run my test app with
--
http://mail.python.org/mailman/listinfo/python-list
Re: using only the django ORM (DB access model) and nothing else.
On 06/22/2011 03:04 AM, Ian Kelly wrote: So I need at least a little more to make my script work. There's a bit of magic in the way Django finds things, and I think you'll still need to keep the basic structure of a Django project -- models should be in a models.py file located in an app package, which should be included in the INSTALLED_APPS setting. You just won't have any views or urlconfs or templates or admin sites or anything like that. Hi Ian, Thanks for your answer. Ourt messages crossed. I had exactly the same idea and started playing.. and you are right. The settings module needs only DATABASES and INSTALLED_APPS with one app and in the apps dir I need apart from the compulsory __init__.py only models.py Cool -- http://mail.python.org/mailman/listinfo/python-list
Re: using only the django ORM (DB access model) and nothing else.
On 06/22/2011 03:02 AM, Roy Smith wrote: In article 4e012e8d$0$23682$426a3...@news.free.fr, News123 news1...@free.fr wrote: I don't see any reason you couldn't use the Model layer by itself, if you want to. It pretty much stands on its own. Thanks a lot for confirming, I have now my small example. Just wanted to be sure I don't overlook some tiny, but really annoying detail which would strongly advise against using the model outside of a web framework. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using django ORM from web browser and from command line apps
On 06/22/2011 03:08 AM, Ian Kelly wrote: On Tue, Jun 21, 2011 at 6:21 PM, News123 news1...@free.fr wrote: Out of curiousity: Do you know whether the imports would be executed for each potential command as soon as I call manage.py or only 'on demand'? Off the top of my head, I don't know. Never mind. WIill jsut add a print statement when I make my first test. This should clearly show whether the module is imported all the time or only on demand. -- http://mail.python.org/mailman/listinfo/python-list
Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
::sigh:: Well, I'm stuck again and it has to do with my get_factors function again, I think. Even with the slight optimization, it's taking forever on 20! (factorial, not excitement) :) It's frustrating because I have the Python right, but I'm getting stuck on the math. The problem: What is the smallest positive number that is evenly divisible by all of the numbers from 1 to 20? Here's the function (it's in the problem3.py file, hence the import below): import math def get_factors(number): factors = [] for n in range(2, int(math.sqrt(number))): if number % n == 0: factors.append(n) factors.append(number // n) return factors And here's my new script for the new exercise: import math from problem3 import get_factors max_num = 20 n = math.factorial(max_num) factors = get_factors(n) div_all = [] for x in factors: for y in range(2, max_num+1): if x % y != 0: break elif y == max_num: div_all.append(x) print(min(div_all)) It could easily be that I'm simply approaching it all wrong. I just thought that maybe using the factorial of the highest number in the range (in this case, 20) would be an easy way of finding which numbers to test. -- http://mail.python.org/mailman/listinfo/python-list
Re: Security test of embedded Python
Chris Angelico ros...@gmail.com writes: users to supply scripts which will then run on our servers... The environment is Python 3.3a0 embedded in C++, running on Linux. This doesn't sound like a bright idea, given the well-known difficulty of sandboxing Python. Geordi http://weegen.home.xs4all.nl/eelis/geordi/ has some interesting examples (C++) you might want to try translating to Python and running on your server. It uses ptrace to control the execution of potentially hostile code. I don't know if any exploits have been found or whether it's still active. Maybe you want to look at Lua. IMHO it's not a very nice language, but I've heard that it's easy to embed and sandbox. -- http://mail.python.org/mailman/listinfo/python-list
Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
John Salerno johnj...@gmail.com writes: It's frustrating because I have the Python right, but I'm getting stuck on the math What is the smallest positive number that is evenly divisible by all of the numbers from 1 to 20? The answer is lcm [1,2,3, ... 20]. You can figure out how to implement lcm. The Euler problems are not really programming exercises. They are exercises in math and algorithms. Quite a lot of them involve thinking clever and fast ways to do stuff that would be trivial (but too slow) by brute force. In general, once you figure out the right algorithm, writing the code is easy. But you have to be fairly mathematically attuned, to have any chance of spotting the algorithm. If you want programming exercises that are less mathematical, there are some nice ones at rubyquiz.com. They are intended for Ruby but of course you can solve them in Python. -- http://mail.python.org/mailman/listinfo/python-list
Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
On 22/06/2011 02:21, John Salerno wrote: ::sigh:: Well, I'm stuck again and it has to do with my get_factors function again, I think. Even with the slight optimization, it's taking forever on 20! (factorial, not excitement) :) It's frustrating because I have the Python right, but I'm getting stuck on the math. The problem: What is the smallest positive number that is evenly divisible by all of the numbers from 1 to 20? You don't need factorials, just remember that each of the numbers can be expressed as the product of a multiset of prime factors. -- http://mail.python.org/mailman/listinfo/python-list
Re: Security test of embedded Python
On Wed, Jun 22, 2011 at 12:02 PM, Paul Rubin no.email@nospam.invalid wrote: Chris Angelico ros...@gmail.com writes: users to supply scripts which will then run on our servers... The environment is Python 3.3a0 embedded in C++, running on Linux. This doesn't sound like a bright idea, given the well-known difficulty of sandboxing Python. So it seems! Less than half an hour after I made the announcement post, the box had been compromised. Geordi http://weegen.home.xs4all.nl/eelis/geordi/ has some interesting examples (C++) you might want to try translating to Python and running on your server. It uses ptrace to control the execution of potentially hostile code. I don't know if any exploits have been found or whether it's still active. Thanks, will look into it. Maybe you want to look at Lua. IMHO it's not a very nice language, but I've heard that it's easy to embed and sandbox. Yeah, I've used Lua before (in a game called Angband), and it's not that great. But security's more important than ideal language syntax. I'll also be looking into Pike. Unfortunately its community is far smaller than Python's, so security holes may be less obvious. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Security test of embedded Python
Followup: The test box has been administratively taken offline after about an hour of testing. Thank you to everyone who participated; it seems we have a lot of changes to make! Monty failed the test. But it was an incredibly successful test. And hopefully, we'll be bringing things back online for another shot once things are sorted out! Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Security test of embedded Python
Chris Angelico ros...@gmail.com writes: I'll also be looking into Pike. Unfortunately its community is far smaller than Python's, so security holes may be less obvious. Actually the most obvious and widespread sandboxed language these days is Javascript. There's several embeddable implementations. Maybe you should just use one of those. -- http://mail.python.org/mailman/listinfo/python-list
Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?
John Salerno wrote: ::sigh:: Well, I'm stuck again and it has to do with my get_factors function again, I think. Even with the slight optimization, it's taking forever on 20! (factorial, not excitement) :) It's frustrating because I have the Python right, but I'm getting stuck on the math. The problem: What is the smallest positive number that is evenly divisible by all of the numbers from 1 to 20? Here's the function (it's in the problem3.py file, hence the import below): import math def get_factors(number): factors = [] for n in range(2, int(math.sqrt(number))): if number % n == 0: factors.append(n) factors.append(number // n) return factors And here's my new script for the new exercise: import math from problem3 import get_factors max_num = 20 n = math.factorial(max_num) factors = get_factors(n) div_all = [] for x in factors: for y in range(2, max_num+1): if x % y != 0: break elif y == max_num: div_all.append(x) print(min(div_all)) It could easily be that I'm simply approaching it all wrong. I just thought that maybe using the factorial of the highest number in the range (in this case, 20) would be an easy way of finding which numbers to test. These are almost trick questions in a way, because of the math behind them. If the question were What is the tallest high-school student in Scranton, PA? then searching a population for the property would be the only way to go. BUT you can also build up the answer knowing the factorization of all the numbers up to 20. Mel. -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I speed up a script that iterates over a large range (600 billion)?
On Tue, Jun 21, 2011 at 4:30 PM, Terry Reedy tjre...@udel.edu wrote: On 6/21/2011 3:48 PM, John Salerno wrote: Absolutely not! Each problem has been designed according to a one- minute rule, which means that although it may take several hours to design a successful algorithm with more difficult problems, an efficient implementation will allow a solution to be obtained on a modestly powered computer in less than one minute. That statement is for C, not Python. Python is efficient with human time, but not machine time. If something really takes a minute in C, allow yourself at least 10 minutes or even more with plain CPython. -- Terry Jan Reedy Python is the second most popular language on Project Euler, at 14358 users compared to 15897 who use C/C++. I'm pretty sure they don't assume you use C. Although Python's longs do make some of the early problems really really easy. -- http://mail.python.org/mailman/listinfo/python-list
Re: Security test of embedded Python
On Tue, Jun 21, 2011 at 7:40 PM, Paul Rubin no.email@nospam.invalid wrote: Chris Angelico ros...@gmail.com writes: I'll also be looking into Pike. Unfortunately its community is far smaller than Python's, so security holes may be less obvious. Actually the most obvious and widespread sandboxed language these days is Javascript. There's several embeddable implementations. Maybe you should just use one of those. Use Pyjamas with that and now you have your sandboxed Python :) -- http://mail.python.org/mailman/listinfo/python-list
