[issue24159] Misleading TypeError when pickling bytes to a file opened as text
Changes by Jon Clements jon...@googlemail.com: -- nosy: +joncle ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24159 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23864] issubclass without registration only works for one-trick pony collections ABCs.
Changes by Jon Clements jon...@googlemail.com: -- nosy: +joncle ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23864 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18925] select.poll.modify is not documented
Jon Clements added the comment: Was looking up epoll.modify and noticed in the docs it's listed as Modify a register file descriptor. - I believe that should be Modify a registered file descriptor... -- nosy: +joncle ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18925 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19363] Python 2.7's future_builtins.map is not compatible with Python 3's map
Changes by Jon Clements jon...@googlemail.com: -- nosy: +joncle ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19363 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15136] Decimal accepting Fraction
New submission from Jon Clements jon...@googlemail.com:
I'm not a numeric expert but I was looking at a post on S/O which related to
converting a Fraction to a certain amount of decimal places. I've had a hunt on
the tracker but couldn't find anything relevant, but if I've missed it, I
apologise.
# F=Fraction, D=Decimal classes
If I try num = D( F(5, 7) )
I get: TypeError: Cannot convert Fraction(5, 7) to Decimal
So I do:
D(f.numerator) / D(f.denominator)
Decimal('0.7142857142857142857142857143')
Which I think is the correct result?
I guess my question is - should Decimal do this implicitly for Fraction?
--
components: Library (Lib)
messages: 163397
nosy: joncle
priority: normal
severity: normal
status: open
title: Decimal accepting Fraction
type: enhancement
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15136
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15136] Decimal accepting Fraction
Jon Clements jon...@googlemail.com added the comment: Mark - I bow to your superiour knowledge here. However, would not a classmethod of .from_fraction be welcome? ie, I could write: d = D.from_fraction(5, 7) Then the documents labour the point about what you've mentioned? Just an idea, but fully realise you're the man best suited to decide, so I'll be happy with whatever you say. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15136 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15136] Decimal accepting Fraction
Jon Clements jon...@googlemail.com added the comment: Not sure what's going on with my machine today: keep sending things to early. I meant: D.from_fraction(F) where if F is not of type Fraction, then the args are used to construct a Fraction - so can use an existing or create one. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15136 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15136] Decimal accepting Fraction
Jon Clements jon...@googlemail.com added the comment:
The more I think about this - the shades of grey kick in.
D.from_fraction(F or creatable F)
Then it would be 'reasonable to assume' for a F.to_decimal() to exist.
Possibly with an optional context argument.
Then, what happens if I do D('2.45') * F(24 / 19)...
I'll leave it to the experts, but have noticed Raymond has ideas for this, but
not yet commented?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15136
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Is that safe to use ramdom.random() for key to encrypt?
On Sun, 17 Jun 2012 23:17:37 +, Steven D'Aprano wrote: On Mon, 18 Jun 2012 08:41:57 +1000, Chris Angelico wrote: On Mon, Jun 18, 2012 at 3:06 AM, Rafael Durán Castañeda rafadurancastan...@gmail.com wrote: The language Python includes a SystemRandom class that obtains cryptographic grade random bits from /dev/urandom on a Unix-like system, including Linux and Mac OS X, while on Windows it uses CryptGenRandom. /dev/urandom isn't actually cryptographically secure; it promises not to block, even if it has insufficient entropy. But in your instance... Correct. /dev/random is meant to be used for long-lasting cryptographically-significant uses, such as keys. urandom is not. http://en.wikipedia.org/wiki//dev/random Do you think is secure enough for token generation? (40 chars long tokens are used for password reset links in a website, there isn't any special security concern for the web). ... it probably is fine, since password reset tokens don't need to be as secure as encryption keys (if anyone _does_ figure out how to predict your password resets, all they'll be able to do is lock people out of their accounts one by one, not snoop on them all unbeknownst, and you'll be able to see log entries showing the resets - you DO log them, right?). In fact, you could probably get away with something pretty trivial there, like a SHA1 of the current timestamp, the user name, and the user's current password hash. The chances that anybody would be able to exploit that are fairly low, given that you're not a bank or other high-profile target. If I were an identity thief, I would *love* low-profile targets. Even though the payoff would be reduced, the cost would be reduced even more: - they tend to be complacent, even more so than high-profile targets; - they tend to be smaller, with fewer resources for security; - mandatory disclosure laws tend not to apply to them; - they don't tend to have the resources to look for anomalous usage patterns, if they even cared enough to want to. If there was a Facebook-like website that wasn't Facebook[1], but still with multiple tens of thousands of users, I reckon a cracker who didn't vandalise people's accounts could steal private data from it for *years* before anyone noticed, and months or years more before they did something about it. [1] And very likely a Facebook-like website that *was* Facebook. I reckon the odds are about 50:50 that FB would prefer to keep a breach secret than risk the bad publicity by fixing it. -- Steven I'm reminded of: http://xkcd.com/936/ http://xkcd.com/792/ There's also one where it's pointed out it's easier to brute force a person who has the code, than brute force the computer. [but can't find that one at the moment] -- http://mail.python.org/mailman/listinfo/python-list
Re: Is that safe to use ramdom.random() for key to encrypt?
On Sun, 17 Jun 2012 12:31:04 +1000, Chris Angelico wrote: On Sun, Jun 17, 2012 at 12:15 PM, Yesterday Paid howmuchisto...@gmail.com wrote: I'm making cipher program with random.seed(), random.random() as the key table of encryption. I'm not good at security things and don't know much about the algorithm used by random module. For security, you don't want any algorithm, you want something like /dev/random (on Unix-like platforms). I'm pretty sure Python includes crypto facilities. Unless it (most oddly) lacks these batteries, I would recommend using one of them instead. ChrisA Cryptography is a complex subject - I've had the (mis)fortune to study it briefly. Whatever you do - *do not* attempt to write your own algorithm. Python includes hashlib (forms of SHA and MD5) and uuid modules, but I take it a symmetric or possibly public/private key system is required - depending on what you want to secure, where it's stored and who needs access. I generally find a separate partition with an encrypted file-system (which is fairly straight forward on *nix systems or I think there's a product out there that works with Windows), is a lot easier and puts the load on the filesystem/OS instead of having to be handled in your application is a lot simpler. Jon -- http://mail.python.org/mailman/listinfo/python-list
Re: file pointer array
On 06/06/12 18:54, Prasad, Ramit wrote:
data= []
for index in range(N, 1): # see Chris Rebert's comment
with open('data%d.txt' % index,'r') as f:
data.append( f.readlines() )
I think data.extend(f) would be a better choice.
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: file pointer array
On 06/06/12 19:51, MRAB wrote:
On 06/06/2012 19:28, Jon Clements wrote:
On 06/06/12 18:54, Prasad, Ramit wrote:
data= []
for index in range(N, 1): # see Chris Rebert's comment
with open('data%d.txt' % index,'r') as f:
data.append( f.readlines() )
I think data.extend(f) would be a better choice.
.extend does something different, and range(N, 1) is an empty range
if N 0.
Mea culpa - I had it in my head the OP wanted to treat the files as one
contiguous one. So yeah:
# something equiv to... (unless it is definitely a fixed range in which
# case (x)range can be used)
data = [ list(open(fname)) for fname in iglob('/home/jon/data*.txt') ]
# then if they ever need to treat it as a contiguous sequence...
all_data = list(chain.from_iterable(data))
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Compare 2 times
On 06/06/12 14:39, Christian Heimes wrote: Am 06.06.2012 14:50, schrieb loial: I have a requirement to test the creation time of a file with the current time and raise a message if the file is more than 15 minutes old. Platform is Unix. I have looked at using os.path.getctime for the file creation time and time.time() for the current time, but is this the best approach? Lots of people are confused by ctime because they think 'c' stands for change. That's wrong. st_ctime is status change time. The ctime is updated when you change (for example) owner or group of a file, create a hard link etc. POSIX has no concept of creation time stamp. Christian I haven't thought this through too much, but perhaps an ugly work-around would be to use inotify (in some kind of daemon) to watch for the IN_CREATE events and store the crtime in a personal DB. Then possibly look at some sort of scheduling to fulfil what happens after 15 minutes. I'm sure there's subtleties I'm missing, but just thought it could be useful. Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: DBF records API
On 01/06/12 23:13, Tim Chase wrote:
On 06/01/12 15:05, Ethan Furman wrote:
MRAB wrote:
I'd probably think of a record as being more like a dict (or an
OrderedDict)
with the fields accessed by key:
record[name]
but:
record.deleted
Record fields are accessible both by key and by attribute -- by key
primarily for those cases when the field name is in a variable:
for field in ('full_name','nick_name','pet_name'):
print record[field]
and since dbf record names cannot start with _ and are at most 10
characters long I've used longer than that method names... but if I want
to support dbf version 7 that won't work.
It seems to me that, since you provide both the indexing notation
and the dotted notation, just ensure that the methods such as
dbf.scatter_fields
*always* trump and refer to the method. This allows for convenience
of using the .field_name notation for the vast majority of cases,
but ensures that it's still possible for the user (of your API) to
use the indexing method to do things like
value = dbf[scatter_fields]
if they have a thusly-named field name and want its value.
-tkc
I did think about *trumping* one way or the other, but both *ugh*.
Ethan:
I think offering both is over-complicating the design for no gain, and
possible complications later. For instance, what if you introduce a
method/property called last to get the last row of a table, it'll
cause some head-scratching as someone will suddenly have to make sure
your API changes didn't conflict with their column names (or if they've
used yours as a base and introduce methods, doesn't interfere with their
users of their version of the library...)
To most developers, I think blah[whatever] is perfectly clear as
looking up a value via key is mostly done that way.
I suppose you could use __getitem__ to grab certain fields in one go (
as per your example - from any iterable that isn't a basestring? - and
users would probably enjoy not keep re-typing record.xxx and would
save you having to invent another possibly conflicting name) such as:
print record['full_name', 'nick_name', 'pet_name'] # looks clean to me
In short I totally agree with MRAB here.
Just my 2p,
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: sqlite INSERT performance
On Thursday, 31 May 2012 16:25:10 UTC+1, duncan smith wrote: On 31/05/12 06:15, John Nagle wrote: On 5/30/2012 6:57 PM, duncan smith wrote: Hello, I have been attempting to speed up some code by using an sqlite database, but I'm not getting the performance gains I expected. SQLite is a lite database. It's good for data that's read a lot and not changed much. It's good for small data files. It's so-so for large database loads. It's terrible for a heavy load of simultaneous updates from multiple processes. Once the table is created the data will not be changed at all. Corresponding integer codes will have to be generated for columns. (I want to do this lazily because some columns might never be needed for output files, and processing all columns was relatively expensive for my initial solution.) After that it's a series of 'SELECT a, b, ... FROM table WHERE f=g ORDER by a, b, ...' style queries dumped to space separated text files. However, wrapping the inserts into a transaction with BEGIN and COMMIT may help. Unfortunately there's no discernible difference. If you have 67 columns in a table, you may be approaching the problem incorrectly. Quite possibly. I have defined start and end points. The data are contained in text files. I need to do the mapping to integer codes and generate output files for subsets of variables conditional on the levels of other variables. (I was doing the subsequent sorting separately, but if I'm using SQL I guess I might as well include that in the query.) The output files are inputs for other (C++) code that I have no control over. Any approach that doesn't consume large amounts of memory will do. Cheers. Duncan It might be worth checking out https://sdm.lbl.gov/fastbit/ which has Python bindings (nb: the library itself takes a while to compile), but I'm not I00% sure it would meet all your requirements. Jon -- http://mail.python.org/mailman/listinfo/python-list
Re: Email Id Verification
On Friday, 25 May 2012 14:36:18 UTC+1, Grant Edwards wrote: On 2012-05-25, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Thu, 24 May 2012 05:32:16 -0700, niks wrote: Hello everyone.. I am new to asp.net... I want to use Regular Expression validator in Email id verification.. Why do you want to write buggy code that makes your users hate your program? Don't do it! Write good code, useful code! Validating email addresses is the wrong thing to do. I have to agree with Steven. Nothing will make your users swear at you as certainly as when you refuse to accept the e-mail address at which the reeive e-mail all day every day. -- Grant Edwards grant.b.edwardsYow! I appoint you at ambassador to Fantasy gmail.comIsland!!! Ditto. This would be my public email, but (like most I believe) also have 'private' and work email addresses. For the OP, just trying to check an email is syntactically correct is okay-ish if done properly. Normally as mentioned you just send a confirmation email to said address with some id and link that confirms (normally with an expiry period). Some mail servers support the does this mailbox exist? request, but I fear these days due to spam, most will just say no -- so the only option is to send and handle a bounce (and some don't even send back bounces). And a pretty good way for malicious people to make mail servers think you're trying a DoS. Although, what I'm finding useful is an option of auth'ing with twitter, facebook, google etc... Doesn't require a huge amount of work, and adds a bit of validity to the request. Jon (who still didn't get any bloody Olympic tickets). -- http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic comparison operators
Any time you find yourself thinking that you want to use eval to solve a problem, take a long, cold shower until the urge goes away. If you have to ask why eval is dangerous, then you don't know enough about programming to use it safely. Scrub it out of your life until you have learned about code injection attacks, data sanitation, trusted and untrusted input. Then you can come back to eval and use it safely and appropriately. I would +1 QOTW - but fear might have to cheat and say +1 to 2 paragraphs of the week :) Jon. -- http://mail.python.org/mailman/listinfo/python-list
usenet reading
Hi All, Normally use Google Groups but it's becoming absolutely frustrating - not only has the interface changed to be frankly impractical, the posts are somewhat random of what appears, is posted and whatnot. (Ironically posted from GG) Is there a server out there where I can get my news groups? I use to be with an ISP that hosted usenet servers, but alas, it's no longer around... Only really interested in Python groups and C++. Any advice appreciated, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: A question of style (finding item in list of tuples)
On Monday, 21 May 2012 13:37:29 UTC+1, Roy Smith wrote:
I've got this code in a django app:
CHOICES = [
('NONE', 'No experience required'),
('SAIL', 'Sailing experience, new to racing'),
('RACE', 'General racing experience'),
('GOOD', 'Experienced racer'),
('ROCK', 'Rock star'),
]
def experience_text(self):
for code, text in self.CHOICES:
if code == self.level:
return text
return
Calling experience_text(ROCK) should return Rock star. Annoyingly,
django handles this for you automatically inside a form, but if you also
need it in your application code, you have to roll your own.
The above code works, but it occurs to me that I could use the much
shorter:
def experience_text(self):
return dict(CHOICES).get(self.level, ???)
So, the question is, purely as a matter of readability, which would you
find easier to understand when reading some new code? Assume the list
of choices is short enough that the cost of building a temporary dict on
each call is negligible. I'm just after style and readability here.
Haven't used django in a while, but doesn't the model provide a
get_experience_display() method which you could use...
Failing that, if order isn't important, you can not bother with tuples and have
CHOICES be a dict, then pass choices=CHOICES.iteritems() as I believe it takes
any iterable, and maybe plug an ordereddict if order is important.
hth
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: key/value store optimized for disk storage
On Friday, 4 May 2012 16:27:54 UTC+1, Steve Howell wrote: On May 3, 6:10 pm, Miki Tebeka miki.teb...@gmail.com wrote: I'm looking for a fairly lightweight key/value store that works for this type of problem: I'd start with a benchmark and try some of the things that are already in the standard library: - bsddb - sqlite3 (table of key, value, index key) - shelve (though I doubt this one) Thanks. I think I'm ruling out bsddb, since it's recently deprecated: http://www.gossamer-threads.com/lists/python/python/106494 I'll give sqlite3 a spin. Has anybody out there wrapped sqlite3 behind a hash interface already? I know it's simple to do conceptually, but there are some minor details to work out for large amounts of data (like creating the index after all the inserts), so if somebody's already tackled this, it would be useful to see their code. You might find that for a little effort you get enough out of one of these. Another module which is not in the standard library is hdf5/PyTables and in my experience very fast. Thanks. Could also look at Tokyo cabinet or Kyoto cabinet (but I believe that has slightly different licensing conditions for commercial use). -- http://mail.python.org/mailman/listinfo/python-list
Re: HTML Code - Line Number
On Friday, 27 April 2012 18:09:57 UTC+1, smac...@comcast.net wrote:
Hello,
For scrapping purposes, I am having a bit of trouble writing a block
of code to define, and find, the relative position (line number) of a
string of HTML code. I can pull out one string that I want, and then
there is always a line of code, directly beneath the one I can pull
out, that begins with the following:
td align=left valign=top class=body_cols_middle
However, because this string of HTML code above is not unique to just
the information I need (which I cannot currently pull out), I was
hoping there is a way to effectively say if you find the html string
_ in the line of HTML code above, and the string td align=left
valign=top class=body_co SMac2347 at comcast.net writes:
Hello,
I am having some difficulty generating the output I want from web
scraping. Specifically, the script I wrote, while it runs without any
errors, is not writing to the output file correctly. It runs, and
creates the output .txt file; however, the file is blank (ideally it
should be populated with a list of names).
I took the base of a program that I had before for a different data
gathering task, which worked beautifully, and edited it for my
purposes here. Any insight as to what I might be doing wrote would be
highly appreciated. Code is included below. Thanks!
[quoting reply to first thread]
I would approach it like this...
import lxml.html
QUERY = '//tr[@bgcolor=#F1F3F4][td[starts-with(@class, body_cols)]]'
url = 'http://www.skadden.com/Index.cfm?contentID=44alphaSearch=A'
tree = lxml.html.parse(url).getroot()
trs = tree.xpath(QUERY)
for tr in trs:
tds = [el.text_content() for el in tr.iterfind('td')]
print tds
hth
Jon.
[/quote]
following, then pull everything that follows this second string.
Any thoughts as to how to define a function to do this, or do this
some other way? All insight is much appreciated! Thanks.
SMac2347 at comcast.net writes:
Hello,
[snip]
Any thoughts as to how to define a function to do this, or do this
some other way? All insight is much appreciated! Thanks.
[quote in reply to second thread]
Did you not see my reply to your previous thread?
And why do you want the line number?
[/quote]
I'm trying this on GG, as the mailing list gateway one or t'other does nee seem
to work (mea culpa no doubt).
So may have obscured the issue more with my quoting and snipping, or what not.
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: HTML Code - Line Number
SMac2347 at comcast.net writes: Hello, [snip] Any thoughts as to how to define a function to do this, or do this some other way? All insight is much appreciated! Thanks. Did you not see my reply to your previous thread? And why do you want the line number? Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Web Scraping - Output File
SMac2347 at comcast.net writes:
Hello,
I am having some difficulty generating the output I want from web
scraping. Specifically, the script I wrote, while it runs without any
errors, is not writing to the output file correctly. It runs, and
creates the output .txt file; however, the file is blank (ideally it
should be populated with a list of names).
I took the base of a program that I had before for a different data
gathering task, which worked beautifully, and edited it for my
purposes here. Any insight as to what I might be doing wrote would be
highly appreciated. Code is included below. Thanks!
I would approach it like this...
import lxml.html
QUERY = '//tr[@bgcolor=#F1F3F4][td[starts-with(@class, body_cols)]]'
url = 'http://www.skadden.com/Index.cfm?contentID=44alphaSearch=A'
tree = lxml.html.parse(url).getroot()
trs = tree.xpath(QUERY)
for tr in trs:
tds = [el.text_content() for el in tr.iterfind('td')]
print tds
hth
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Using arguments in a decorator
On Saturday, 21 April 2012 09:25:40 UTC+1, Steven D#39;Aprano wrote:
On Fri, 20 Apr 2012 09:10:15 -0700, Jon Clements wrote:
But I don't know how. I know that I can see the default arguments of
the original function using func.__defaults__, but without knowing the
number and names of func's positional arguments (which I don't know how
to find out) this doesn't help me. Any suggestions?
Possibly take a look at functools.lru_cache (which is Python 3.2+), and
use the code from that (at it's part of the stdlib, someone must have
done design and testing on it!).
With respect Jon, did you read the Original Poster's question closely?
Using a LRU cache doesn't even come close to fixing his problem, which
occurs *before* you do the lookup in the cache.
I did indeed Steven - what I was suggesting was that functools.lru_cache would
be a good starting point. Although I will completely admit that I didn't read
the code for functools.lru_cache thoroughly enough to realise it wouldn't be
suitable for the OP (ie, it sounded right, looked okay at a glance, and I
figured it wasn't a totally unreasonable assumption of a suggestion - so guess
I fell into the old 'assume' trap! [not the first time, and won't be the last
for sure!])
Rotwang's problem is that if you have a function with default arguments:
def func(spam=42):
return result_of_time_consuming_calculation()
then these three function calls are identical and should (but don't)
share a single cache entry:
func()
func(42)
func(spam=42)
The OP would like all three to share a single cache entry without needing
two redundant calculations, which take a long time.
The problem is that the three calls give three different patterns of args
and kwargs:
(), {}
(42,) {}
(), {'spam': 42}
hence three different cache entries, two of which are unnecessary.
I'm wondering if it wouldn't be unreasonable for lru_cache to handle this.
Cheers, Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newbie, homework help, please.
On Saturday, 21 April 2012 18:35:26 UTC+1, someone wrote:
On Saturday, April 21, 2012 12:28:33 PM UTC-5, someone wrote:
Ok, this is my dillema, not only am I new to this programming buisness,
before the last few days, I did not even know what python was, and besides
opening up the internet or word documents, that is most of what I know.
Yet, I have a professor who should be on Psych medication for giving us 3
projects, 2 of which I have not listed here to do. I was able to do
research over the last 3 days, and I have spent 3 days on this project, by
borrowing others ideas on this project. Below, you will find my professors
assignment (oh, and due in one week right before finals, so I am stressing
out so much, cause I don't know why he is crazy enough to assign crap like
this a week before finals when I have Calculus final,chem final, etc. I
have figured out most of the assignment, and below, it will be posted after
the teacher's post of the assignment. What I need help with, and I have
tried relentlessly to find, is how to put freaking stars(asterisks) as
border around a list without installing any other program to a portable
python, of course, this is where my problem lies. Below, you will see what
I have done, please, help!!!
You are required to complete and submit the following programming projects
in Python by the indicated deadline:
Standard Header Information project (5 pts):
Write a program that will:
1) Ask the user for the following information:
- name of file to be created for storing SHI
- user’s name (as part of SHI)
- user’s course and section (as part of SHI)
- user’s semester and year (as part of SHI)
- user’s assignment title (as part of SHI)
2) Write the above SHI data to a text (.txt) file with the name chosen by
the user (above)
3) Close the file that the SHI data was written to
4) Open the file with the SHI data (again)
5) Read the data into different (from part 1) variable names
6) Display the SHI data read from the file in the interpreter with a border
around the SHI data (include a buffer of 1 line/space between the border
and SHI data). An example might look like:
***
* *
* First Name and Last *
* ENGR 109-X *
* Fall 2999 *
* Format Example *
* *
***
textfile=input('Hello, we are about to create a text file. An example would
be: (sample.txt) without the parenthesis. What ever you do name it, it
needs to end in (.txt). What would you like to name your textfile?')
userinput=[input('What is your name?'),input('What is your Course Section
and Course number?'),input('What is the Semester and year?'),input('What is
the title of this class assignment?')]
for item in userinput:
openfile=open(textfile,'w');openfile.writelines(%s\n % item for item
in userinput);openfile.close()
x=textfile;indat=open(x,'r');SHI=indat.read()
def border(Sullivan):
string=SHI
stringlength=len(string)
stringlength=stringlength(%s\n % item for item in stringlength) + 2 *
(3 + 3)
hBorder=stringlength//2** +*[:stringlength%2]
spacer=*+ *(stringlength - 2)+*
fancyText=* +string+ *
return(hBorder,spacer,fancyText,hBorder)
textTuple = border(SHI)
for lines in textTuple:
print (lines)
almost forgot, it has to have a 1 inch border around the top, bottom, left,
and right, with it being aligned to the left. In the picture above, that is
not how it actually looks, the stars to the right are aligned on the right,
not right next to each other. Thanks.
Honestly phrased question - well done.
Look at the textwrap module - I have no idea how you'll got an inch outputting
in just text, as I might have a slightly different font setting and logical and
physical inches are different.
Good luck, Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Using arguments in a decorator
On Friday, 20 April 2012 16:57:06 UTC+1, Rotwang wrote:
Hi all, here's a problem I don't know how to solve. I'm using Python 2.7.2.
I'm doing some stuff in Python which means I have cause to call
functions that take a while to return. Since I often want to call such a
function more than once with the same arguments, I've written a
decorator to eliminate repeated calls by storing a dictionary whose
items are arguments and their results:
def memo(func):
def memofunc(*args, **kwargs):
twargs = tuple(kwargs.items())
if (args, twargs) in memofunc.d:
return copy(memofunc.d[(args, twargs)])
memofunc.d[(args, twargs)] = func(*args, **kwargs)
return copy(memofunc.d[(args, twargs)])
memofunc.__name__ = func.__name__
memofunc.d = {}
return memofunc
If a function f is decorated by memo, whenever f is called with
positional arguments args and keyword arguments kwargs, the decorated
function defines twargs as a hashable representation of kwargs, checks
whether the tuple (args, twargs) is in f's dictionary d, and if so
returns the previously calculated value; otherwise it calculates the
value and adds it to the dictionary (copy() is a function that returns
an object that compares equal to its argument, but whose identity is
different - this is useful if the return value is mutable).
As far as I know, the decorated function will always return the same
value as the original function. The problem is that the dictionary key
stored depends on how the function was called, even if two calls should
be equivalent; hence the original function gets called more often than
necessary. For example, there's this:
@memo
def f(x, y = None, *a, **k):
return x, y, a, k
f(1, 2)
(1, 2, (), {})
f.d
{((1, 2), ()): (1, 2, (), {})}
f(y = 2, x = 1)
(1, 2, (), {})
f.d
{((1, 2), ()): (1, 2, (), {}), ((), (('y', 2), ('x', 1))): (1, 2, (), {})}
What I'd like to be able to do is something like this:
def memo(func):
def memofunc(*args, **kwargs):
#
# define a tuple consisting of values for all named positional
# arguments occurring in the definition of func, including
# default arguments if values are not given by the call, call
# it named
#
# define another tuple consisting of any positional arguments
# that do not correspond to named arguments in the definition
# of func, call it anon
#
# define a third tuple consisting of pairs of names and values
# for those items in kwargs whose keys are not named in the
# definition of func, call it key
#
if (named, anon, key) in memofunc.d:
return copy(memofunc.d[(named, anon, key)])
memofunc.d[(named, anon, key)] = func(*args, **kwargs)
return copy(memofunc.d[(named, anon, key)])
memofunc.__name__ = func.__name__
memofunc.d = {}
return memofunc
But I don't know how. I know that I can see the default arguments of the
original function using func.__defaults__, but without knowing the
number and names of func's positional arguments (which I don't know how
to find out) this doesn't help me. Any suggestions?
--
Hate music? Then you'll hate this:
http://tinyurl.com/psymix
Possibly take a look at functools.lru_cache (which is Python 3.2+), and use the
code from that (at it's part of the stdlib, someone must have done design and
testing on it!). http://hg.python.org/cpython/file/default/Lib/functools.py
Jon
--
http://mail.python.org/mailman/listinfo/python-list
Re: Regular expressions, help?
On Thursday, 19 April 2012 07:11:54 UTC+1, Sania wrote:
Hi,
So I am trying to get the number of casualties in a text. After 'death
toll' in the text the number I need is presented as you can see from
the variable called text. Here is my code
I'm pretty sure my regex is correct, I think it's the group part
that's the problem.
I am using nltk by python. Group grabs the string in parenthesis and
stores it in deadnum and I make deadnum into a list.
text=accounts put the death toll at 637 and those missing at
653 , but the total number is likely to be much bigger
dead=re.match(r.*death toll.*(\d[,\d\.]*), text)
deadnum=dead.group(1)
deaths.append(deadnum)
print deaths
Any help would be appreciated,
Thank you,
Sania
Or just don't fully rely on a regex. I would, for time, and the little sanity I
believe I have left, would just do something like:
death_toll = re.search(r'death toll.*\d+', text).group().rsplit(' ', 1)[1]
hth,
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: How do you refer to an iterator in docs?
On Thursday, 19 April 2012 13:21:20 UTC+1, Roy Smith wrote: Let's say I have a function which takes a list of words. I might write the docstring for it something like: def foo(words): Foo-ify words (which must be a list) What if I want words to be the more general case of something you can iterate over? How do people talk about that in docstrings? Do you say something which can be iterated over to yield words, an iterable over words, or what? I can think of lots of ways to describe the concept, but most of them seem rather verbose and awkward compared to a list of words, a dictionary whose keys are words, etc. I would just write the function signature as (very similar to how itertools does it): def func(iterable, ..): pass IMHO that documents itself. If you need explicit, look at the itertools documentation. hth Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: escaping
On Monday, 16 April 2012 11:03:31 UTC+1, Kiuhnm wrote:
On 4/16/2012 4:42, Steven D'Aprano wrote:
On Sun, 15 Apr 2012 23:07:36 +0200, Kiuhnm wrote:
This is the behavior I need:
path = path.replace('\\', '')
msg = . {} .. '{}' .. {} ..format(a, path, b)
Is there a better way?
This works for me:
a = spam
b = ham
path = rC:\a\b\c\d\e.txt
msg = . %s .. %r .. %s . % (a, path, b)
print msg
. spam .. 'C:\\a\\b\\c\\d\\e.txt' .. ham .
I like this one. Since I read somewhere that 'format' is preferred over
'%', I was focusing on 'format' and I didn't think of '%'.
Anyway, it's odd that 'format' doesn't offer something similar.
Kiuhnm
If you look at http://docs.python.org/library/string.html#format-string-syntax
you'll notice the equiv. of %r is {!r}
--
http://mail.python.org/mailman/listinfo/python-list
Re: ordering with duck typing in 3.1
On Monday, 9 April 2012 12:33:25 UTC+1, Neil Cerutti wrote: On 2012-04-07, Jon Clements jon...@googlemail.com wrote: Any reason you can't derive from int instead of object? You may also want to check out functions.total_ordering on 2.7+ functools.total_ordering I was temporarily tripped up by the aforementioned documentation, myself. -- Neil Cerutti Oops. I sent it from a mobile tablet device - I got auto-corrected. But yes, it is functools.total_ordering - TY you Neil. Jon. -- http://mail.python.org/mailman/listinfo/python-list
ordering with duck typing in 3.1
Any reason you can't derive from int instead of object? You may also want to check out functions.total_ordering on 2.7+ -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Gotcha's?
On Wednesday, 4 April 2012 23:34:20 UTC+1, Miki Tebeka wrote:
Greetings,
I'm going to give a Python Gotcha's talk at work.
If you have an interesting/common Gotcha (warts/dark corners ...) please
share.
(Note that I want over http://wiki.python.org/moin/PythonWarts already).
Thanks,
--
Miki
One I've had to debug...
text = 'abcdef'
if text.find('abc'):
print 'found it!'
# Nothing prints as bool(0) is False
if text.find('bob'):
print 'found it!'
found it!
Someone new who hasn't read the docs might try this, but then I guess it's not
really a gotcha if they haven't bothered doing that.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Async IO Server with Blocking DB
On Tuesday, 3 April 2012 23:13:24 UTC+1, looking for wrote: Hi We are thinking about building a webservice server and considering python event-driven servers i.e. Gevent/Tornado/ Twisted or some combination thereof etc. We are having doubts about the db io part. Even with connection pooling and cache, there is a strong chance that server will block on db. Blocking for even few ms is bad. can someone suggest some solutions or is async-io is not at the prime- time yet. Thanks Maybe look at Cyclone (a Tornado variation built on Twisted), and various modules that will offer synch and events - GIYF! It's doable! Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Best way to structure data for efficient searching
On Wednesday, 28 March 2012 19:39:54 UTC+1, larry@gmail.com wrote: I have the following use case: I have a set of data that is contains 3 fields, K1, K2 and a timestamp. There are duplicates in the data set, and they all have to processed. Then I have another set of data with 4 fields: K3, K4, K5, and a timestamp. There are also duplicates in that data set, and they also all have to be processed. I need to find all the items in the second data set where K1==K3 and K2==K4 and the 2 timestamps are within 20 seconds of each other. I have this working, but the way I did it seems very inefficient - I simply put the data in 2 arrays (as tuples) and then walked through the entire second data set once for each item in the first data set, looking for matches. Is there a better, more efficient way I could have done this? It might not be more *efficient* but others might find it more readable, and it'd be easier to change later. Try an in-memory SQL DB (such as sqlite3) and query as (untested) select t2.* from t1 join t2 on k1=k3 and k2=k4 where abs(t1.timestamp - t2.timestamp) 20 Failing that, two (default)dicts with a tuple as the pair, then use that as your base. Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: help with subclassing problem
On Thursday, 29 March 2012 21:23:20 UTC+1, Peter wrote: I am attempting to subclass the date class from the datetime package. Basically I want a subclass that can take the date as a string (in multiple formats), parse the string and derive the year,month and day information to create a date instance i.e. class MyDate(datetime.date): def __init__(self, the_date): # magic happens here to derives year, month and day from the_date datetime.date.__init__(self, year, month, day) But no matter what I do, when I attempt to create an instance of the new class, I get the error message: Traceback (most recent call last): File stdin, line 1, in module TypeError: Required argument 'year' (pos 1) not found I have even created a class which doesn't include the argument I want to use but with default arguments i.e. class MyDate (datetime.date): def __init__(self, year = 1, month = 1, day = 1): datetime.date.__init__(self, year, month, day) and get the same error message. What am I doing wrong here? Thanks for any help, Peter Details here: http://stackoverflow.com/questions/399022/why-cant-i-subclass-datetime-date Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Data mining/pattern recogniton software in Python?
On Friday, 23 March 2012 16:43:40 UTC, Grzegorz Staniak wrote: Hello, I've been asked by a colleague for help in a small educational project, which would involve the recognition of patterns in a live feed of data points (readings from a measuring appliance), and then a more general search for patterns on archival data. The language of preference is Python, since the lab uses software written in Python already. I can see there are packages like Open CV, scikit-learn, Orange that could perhaps be of use for the mining phase -- and even if they are slanted towards image pattern recognition, I think I'd be able to find an appropriate package for the timeseries analyses. But I'm wondering about the live phase -- what approach would you suggest? I wouldn't want to force an open door, perhaps there are already packages/modules that could be used to read data in a loop i.e. every 10 seconds, maintain a a buffer of 15 readings and ring a bell when the data in buffer form a specific pattern (a spike, a trough, whatever)? I'll be grateful for a push in the right direction. Thanks, GS -- Grzegorz Staniak gstaniak _at_ gmail [dot] com It might also be worth checking out pandas[1] and scikits.statsmodels[2]. In terms of reading data in a loop I would probably go for a producer-consumer model (possibly using a Queue[3]). Have the consumer constantly try to get another reading, and notify the consumer which can then determine if it's got enough data to calculate a peak/trough. This article is also a fairly good read[4]. That's some pointers anyway, hth, Jon. [1] http://pandas.pydata.org/ [2] http://statsmodels.sourceforge.net/ [3] http://docs.python.org/library/queue.html [4] http://www.laurentluce.com/posts/python-threads-synchronization-locks-rlocks-semaphores-conditions-events-and-queues/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Fetching data from a HTML file
On Friday, 23 March 2012 13:52:05 UTC, Sangeet wrote:
Hi,
I've got to fetch data from the snippet below and have been trying to match
the digits in this to specifically to specific groups. But I can't seem to
figure how to go about stripping the tags! :(
trtd align=centerbSum/b/tdtd/tdtd align='center'
class=green245/tdtd align='center' class=red11/tdtd
align='center'0/tdtd align='center' 256/tdtd align='center' 1.496
[min]/td/tr
/table
Actually, I'm working on ROBOT Framework, and haven't been able to figure out
how to read data from HTML tables. Reading from the source, is the best (read
rudimentary) way I could come up with. Any suggestions are welcome!
Thanks,
Sangeet
I would personally use lxml - a quick example:
# -*- coding: utf-8 -*-
import lxml.html
text =
trtd align=centerbSum/b/tdtd/tdtd align='center'
class=green245/tdtd align='center' class=red11/tdtd
align='center'0/tdtd align='center' 256/tdtd align='center' 1.496
[min]/td/tr
/table
table = lxml.html.fromstring(text)
for tr in table.xpath('//tr'):
print [ (el.get('class', ''), el.text_content()) for el in
tr.iterfind('td') ]
[('', 'Sum'), ('', ''), ('green', '245'), ('red', '11'), ('', '0'), ('',
'256'), ('', '1.496 [min]')]
It does a reasonable job, but if it doesn't work quite right, then there's a
.fromstring(parser=...) option, and you should be able to pass in ElementSoup
and try your luck from there.
hth,
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable (OT)
On Thursday, 22 March 2012 08:56:17 UTC, Steven D#39;Aprano wrote: On Wed, 21 Mar 2012 18:35:16 -0700, Steve Howell wrote: On Mar 21, 11:06 am, Nathan Rice nathan.alexander.r...@gmail.com wrote: [snip]. Different programming languages are good for different things because they have been designed to work in different problem/solution spaces. Although I dislike C with a passion, I do recognise that it is good for when the programmer needs fine control over the smallest details. It is, after all, a high-level assembler. Likewise for Forth, which lets you modify the compiler and language as you go. Some languages are optimized for the compiler, some for the writer, and some for the reader. So are optimized for numeric work, others for database access. Some are Jack-Of-All-Trades. Each language encourages its own idioms and ways of thinking about programming. When it comes to programming, I say, let a thousand voices shout out. Instead of imagining a single language so wonderful that every other language is overshadowed and forgotten, imagine that the single language is the next Java, or C, or even for that matter Python, but whatever it is, it's not ideal for the problems you care about, or the way you think about them. Not so attractive now, is it? The optimistic view is that there will be some kind of inflection point around 2020 or so. I could imagine a perfect storm of good things happening, like convergence on a single browser platform, You call that a perfect storm of good things. I call that sort of intellectual and software monoculture a nightmare. I want a dozen browsers, not one of which is so common that web designers can design for it and ignore the rest, not one browser so common that nobody dares try anything new. nearly complete migration to Python 3, further maturity of JVM-based languages, etc., where the bar gets a little higher from what people expect from languages. Instead of fighting semicolons and braces, we start thinking bigger. It could also be some sort of hardware advance, like screen resolutions that are so amazing they let us completely rethink our views on terseness, punctuation, code organization, etc. And what of those with poor eyesight, or the blind? Are they to be excluded from your bigger brave new world? -- Steven On Thursday, 22 March 2012 08:56:17 UTC, Steven D#39;Aprano wrote: On Wed, 21 Mar 2012 18:35:16 -0700, Steve Howell wrote: On Mar 21, 11:06 am, Nathan Rice nathan.alexander.r...@gmail.com wrote: As for syntax, we have a lot of real domain specific languages, such as English, math and logic. They are vetted, understood and useful outside the context of programming. We should approach the discussion of language syntax from the perspective of trying to define a unified syntactical structure for real these DSLs. Ideally it would allow representation of things in a familiar way where possible, while providing an elegant mechanism for descriptions that cut across domains and eliminating redundancy/ambiguity. This is clearly possible, though a truly successful attempt would probably be a work of art for the ages. If I'm reading you correctly, you're expressing frustration with the state of language syntax unification in 2012. You mention language in a broad sense (not just programming languages, but also English, math, logic, etc.), but even in the narrow context of programming languages, the current state of the world is pretty chaotic. And this is a good thing. Programming languages are chaotic because the universe of programming problems is chaotic, and the strategies available to solve those problems are many and varied. Different programming languages are good for different things because they have been designed to work in different problem/solution spaces. Although I dislike C with a passion, I do recognise that it is good for when the programmer needs fine control over the smallest details. It is, after all, a high-level assembler. Likewise for Forth, which lets you modify the compiler and language as you go. Some languages are optimized for the compiler, some for the writer, and some for the reader. So are optimized for numeric work, others for database access. Some are Jack-Of-All-Trades. Each language encourages its own idioms and ways of thinking about programming. When it comes to programming, I say, let a thousand voices shout out. Instead of imagining a single language so wonderful that every other language is overshadowed and forgotten, imagine that the single language is the next Java, or C, or even for that matter Python, but whatever it is, it's not ideal for the problems you care about, or the way you think about them. Not so attractive now, is it? The optimistic view is that there will be some kind of inflection point around 2020 or so. I could
Re: urllib.urlretrieve never returns???
On Monday, 19 March 2012 19:32:03 UTC, Laszlo Nagy wrote: The pythonw.exe may not have the rights to access network resources. Have you set a default timeout for sockets? import socket socket.setdefaulttimeout(10) # 10 seconds I have added pythonw.exe to allowed exceptions. Disabled firewall completely. Set socket timeout to 10 seconds. Still nothing. urllib.urlretrieve does not return from call any other ideas? Maybe try using the reporthook option for urlretrieve, just to see if that does anything... If it constantly calls the hook or never calls it, that's one thing. Alternately, tcpdump/wireshark whatever, to see what the heck is going on with traffic - if any. hth Jon -- http://mail.python.org/mailman/listinfo/python-list
Re: Style question (Poll)
On Wednesday, 14 March 2012 21:16:05 UTC, Terry Reedy wrote: On 3/14/2012 4:49 PM, Arnaud Delobelle wrote: On 14 March 2012 20:37, Croephacroe...@gmail.com wrote: Which is preferred: for value in list: if not value is another_value: value.do_something() break Do you really mean 'is' or '=='? If you mean x is not y, write it that way. 'not x is y' can be misread and misunderstood, depending on whether the 'is' is true or not. not 1 is 1 False not (1 is 1) False (not 1) is 1 False Does not matter how read. not (1 is 0) True (not 1) is 0 False not 1 is 0 True Does matter how read. if list and not list[0] is another_value: list[0].do_something() Or try: value = mylist[0] if value is not another_value: value.dosomething except IndexError: pass I would not do this in this case of index 0, but if the index were a complicated expression or expensive function call, making 'if list' an inadequate test, I might. Hard to say, since they don't do the same thing :) I suspect you meant: for value in list: if not value is another_value: value.do_something() break I always feel uncomfortable with this because it's misleading: a loop that never loops. I agree. Please do not do this in public ;-). -- Terry Jan Reedy I'm not sure it's efficient or even if I like it, but it avoids try/except and the use of a for loop. if next( iter(mylist), object() ) is not another_value: # ... Just my 2p, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to decide if a object is instancemethod?
On Wednesday, 14 March 2012 13:28:58 UTC, Cosmia Luna wrote: class Foo(object): def bar(self): return 'Something' func = Foo().bar if type(func) == type 'instancemethod': # This should be always true pass # do something here What should type at type 'instancemethod'? Thanks Cosmia import inspect if inspect.ismethod(foo): # ... Will return True if foo is a bound method. hth Jon -- http://mail.python.org/mailman/listinfo/python-list
Re: Jinja2 + jQuery tabs widget
On Wednesday, 14 March 2012 14:16:35 UTC, JoeM wrote:
Hi All,
I'm having issues including a {block} of content from Jinja2
template into a jQueryUI tab. Does anyone know if such a thing is
possible? An example is below, which gives me a 500 error when loading
the page.
Thanks,
Joe
script
$(function() {
$( #tabs ).tabs();
});
/script
ul
lia href=#tabs-SumSummary/a/li
lia href=#tabs-MapsMaps/a/li
lia href=#tabs-TablesTables/a/li
lia href=#tabs-AniAnimations/a/li
lia href=#tabs-DefDefinitions/a/li
/ul
{% block map_content %} {% endblock %}
/div
/div
Firstly, this isn't really a Python language question - although jinja2 is a
commonly used module for web frameworks.
Secondly, the code looks fine, except we don't know what's in the map_content
block.
Thirdly, 500 is an internal server error - so it's possible it's nothing to do
with any of this anyway -- could you provide a more comprehensive error message?
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Global join function?
On Wednesday, 14 March 2012 18:41:27 UTC, Darrel Grant wrote: In the virtualenv example bootstrap code, a global join function is used. http://pypi.python.org/pypi/virtualenv subprocess.call([join(home_dir, 'bin', 'easy_install'), 'BlogApplication']) In interpeter, I tried this: [join([], 'bin', 'easy_install')] Traceback (most recent call last): File stdin, line 1, in module NameError: name 'join' is not defined I think I've seen this used elsewhere, but googling only seems to show results about the string method join, not whatever this is. To be clear, I understand how to use .join(list), but have not found any information about this other, seemingly global, join function which takes multiple arguments. It's been bugging me. os.path.join Jon -- http://mail.python.org/mailman/listinfo/python-list
Re: Fast file data retrieval?
On Monday, 12 March 2012 20:31:35 UTC, MRAB wrote: On 12/03/2012 19:39, Virgil Stokes wrote: I have a rather large ASCII file that is structured as follows header line 9 nonblank lines with alphanumeric data header line 9 nonblank lines with alphanumeric data ... ... ... header line 9 nonblank lines with alphanumeric data EOF where, a data set contains 10 lines (header + 9 nonblank) and there can be several thousand data sets in a single file. In addition,*each header has a* *unique ID code*. Is there a fast method for the retrieval of a data set from this large file given its ID code? Probably the best solution is to put it into a database. Have a look at the sqlite3 module. Alternatively, you could scan the file, recording the ID and the file offset in a dict so that, given an ID, you can seek directly to that file position. I would have a look at either bsddb, Tokyo (or Kyoto) Cabinet or hamsterdb. If it's really going to get large and needs a full blown server, maybe MongoDB/redis/hadoop... -- http://mail.python.org/mailman/listinfo/python-list
Re: Finding MIME type for a data stream
On Thursday, 8 March 2012 23:40:13 UTC, Tobiah wrote: I have to assume you're talking python 2, since in python 3, strings cannot generally contain image data. In python 2, characters are pretty much interchangeable with bytes. Yeah, python 2 if you're looking for a specific, small list of file formats, you could make yourself a signature list. Most (not all) formats distinguish themselves in the first few bytes. Yeah, maybe I'll just do that. I'm alowing users to paste images into a rich-text editor, so I'm pretty much looking at .png, .gif, or .jpg. Those should be pretty easy to distinguish by looking at the first few bytes. Pasting images may sound weird, but I'm using a jquery widget called cleditor that takes image data from the clipboard and replaces it with inline base64 data. The html from the editor ends up as an email, and the inline images cause the emails to be tossed in the spam folder for most people. So I'm parsing the emails, storing the image data, and replacing the inline images with an img tag that points to a web2py app that takes arguments that tell it which image to pull from the database. Now that I think of it, I could use php to detect the image type, and store that in the database. Not quite as clean, but that would work. Tobiah Something like the following might be worth a go: (untested) from PIL import Image img = Image.open(StringIO(blob)) print img.format HTH Jon. PIL: http://www.pythonware.com/library/pil/handbook/image.htm -- http://mail.python.org/mailman/listinfo/python-list
Re: os.stat last accessed attribute updating last accessed value
On Feb 4, 9:33 pm, Python_Junkie software.buy.des...@gmail.com wrote: I am trying to obtain the last accessed date. About 50% of the files' attributes were updated such that the file was last accessed when this script touches the file. I was not opening the files Anyone have a thought of why this happened. Python 2.6 on windows xp Read up on NTFS - but on some file systems - to check a file access time is, well umm, is accessing it. Also possible that listing a directory is considered an access. It's the least useful of all records - I've only ever possibly wanted modification or creation times. hth, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Constraints -//- first release -//- Flexible abstract class based validation for attributes, functions and code blocks
On Jan 27, 6:38 am, Nathan Rice nathan.alexander.r...@gmail.com wrote: May I suggest a look at languages such as ATS and Epigram? They use types that constrain values specifically to prove things about your program. Haskell is a step, but as far as proving goes, it's less powerful than it could be. ATS allows you to, at compile-time, declare that isinstance(x, 0 = Symbol() len(L)) for some list L. So it might align well with your ideas. Thanks for the tip. Probably deserves a better name than constraintslib, that makes one think of constraint satisfaction. As you can probably tell from my other projects, I'm bad at coming up with snappy names. I'm bad at doing research on previous projects ;) I guess I'm not plugging my other projects enough... You should check out elementwise. Thanks, Nathan I love elementwise and this one - thanks. If I can be so bold, I would call it 'contracts'. Or, if you want to be more imaginative and esoteric - 'judge'/'barrister'/'solicitor'. Thanks again, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Find the mime type of a file.
On Jan 25, 5:04 pm, Olive di...@bigfoot.com wrote: I want to have a list of all the images in a directory. To do so I want to have a function that find the mime type of a file. I have found mimetypes.guess_type but it only works by examining the extension. In GNU/Linux the file utility do much better by actually looking at the file. Is there an equivalent function in python (as a last resort I can always use the external file utility). Olive You could also try using PIL.(I hardly use it, but...) from PIL import Image for fname in [some list of filenames here]: img = Image.open(fname) print img.format Might be more expensive than the file utility, but that's up to you to determine (open might be lazy, or it might load it - there is a separate load function though, so who knows). hth, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing a serial stream too slowly
On Jan 23, 9:48 pm, M.Pekala mcdpek...@gmail.com wrote: Hello, I am having some trouble with a serial stream on a project I am working on. I have an external board that is attached to a set of sensors. The board polls the sensors, filters them, formats the values, and sends the formatted values over a serial bus. The serial stream comes out like $A1234$$B-10$$C987$, where $A.*$ is a sensor value, $B.*$ is a sensor value, $C.*$ is a sensor value, ect... When one sensor is running my python script grabs the data just fine, removes the formatting, and throws it into a text control box. However when 3 or more sensors are running, I get output like the following: Sensor 1: 373 Sensor 2: 112$$M-160$G373 Sensor 3: 763$$A892$ I am fairly certain this means that my code is running too slow to catch all the '$' markers. Below is the snippet of code I believe is the cause of this problem... def OnSerialRead(self, event): text = event.data self.sensorabuffer = self.sensorabuffer + text self.sensorbbuffer = self.sensorbbuffer + text self.sensorcbuffer = self.sensorcbuffer + text if sensoraenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorabuffer ) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorAValue = s self.sensorabuffer = '' if sensorbenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorbenable) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorBValue = s self.sensorbenable= '' if sensorcenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorcenable) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorCValue = s self.sensorcenable= '' self.DisplaySensorReadings() I think that regex is too slow for this operation, but I'm uncertain of another method in python that could be faster. A little help would be appreciated. You sure that's your code? Your re.search()'s are all the same. -- http://mail.python.org/mailman/listinfo/python-list
Re: can some one help me with my code. thanks
On Jan 20, 9:26 pm, Terry Reedy tjre...@udel.edu wrote:
On 1/20/2012 2:46 PM, Terry Reedy wrote:
On 1/20/2012 1:49 PM, Tamanna Sultana wrote:
can some one help me??
I would like to create a function that, given a bin, which is a list
(example below), generates averages for the numbers separated by a
string 'end'. I am expecting to have 4 averages from the above bin,
since there are 4 sets of numbers separated by 4 'end' strings
[Posting your overly long set of data lines with a '' quote at the
beginning of each line was a nuisance. Reposted with few lines. I will
let you compare your code to mine.]
bin = ['2598.95165', '2541.220308', '221068.0401', 'end', '4834.581952',
'1056.394859', '3010.609563', '2421.437603', '4619.861889',
'3682.012227', '3371.092883', '6651.509488', '7906.092773',
'7297.133447', 'end', '4566.874299', 'end', '4255.700077',
'1857.648393', '11289.48095', '2070.981805', '1817.505094',
'563.0265409', '70796.45356', '565.2123689', '6560.030116',
'2668.934414', '418.666014', '5216.392132', '760.894589', '8072.957639',
'346.5905371', 'end']
def average(bin):
num=[]
total = 0.0
count=0
for number in bin:
if number!='end':
total += float(number)
count+=1
else:
num.append(total/count)
total = 0.0
count= 0
return num
print(average(bin))
[75402.7373526, 4485.0726684, 4566.874299, 7817.36494866]
U're welcome. But do notice Tim's comment. In non-toy situations, you
have to decide how to handle empty collections (return float('nan')?),
or whether to just let whatever happens happen.
If you control the input format, a list of lists would be easier than an
end marker. But sometimes one is handed data and asked to process it as is.
Also note (this is a more advanced topic) that average() could be turned
into a generator function by replacing 'num.append(total/count)' with
'yield total/count' and removing the initialization and return of num.
--
Terry Jan Reedy
Not directing this at you Terry, and you and Tim have made fine points
-- this just appears to me to be the best point at which to respond to
a thread.
To the OP - you have great answers, and, please note this just happens
to be the way I would do this.
I would separate the parsing of the data, and the calculation code
out. I've whipped this up rather quickly, so it might have a few flaws
but...
from itertools import groupby
def partition(iterable, sep=lambda L: L == 'end', factory=float):
for key, vals in groupby(iterable, sep):
if not key: yield map(factory, vals)
# And a pure cheat, but useful if more complex calculations are
required etc... (Plus covers NaN)
import numpy as np
print map(np.mean, partition(bin))
What you've got will work though, so wouldn't worry too much and this
is just my 2p,
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: my new project, is this the right way?
On Nov 14, 10:41 am, Tracubik affdfsdfds...@b.com wrote: Hi all, i'm developing a new program. Mission: learn a bit of database management Idea: create a simple, 1 window program that show me a db of movies i've seen with few (10) fields (actors, name, year etc) technologies i'll use: python + gtk db: that's the question since i'm mostly a new-bye for as regard databases, my idea is to use sqlite at the beginning. Is that ok? any other db to start with? (pls don't say mysql or similar, they are too complex and i'll use this in a second step) is there any general tutorial of how to start developing a database? i mean a general guide to databases you can suggest to me? Thank you all MedeoTL P.s. since i have a ods sheet files (libreoffice calc), is there a way to easily convert it in a sqlite db? (maybe via csv) I would recommend working through the book SQL for Dummies. I found it very clear, and slowly leads you into how to think about design, not just how to manipulate databases. Instead of using Python to start with consider using OOo Base or MS Access (retching noise), so you can use RAD to play with structure and manipulation of your data and create data-entry forms -- this'll allow you to enter data, and play with queries and the structure -- as you won't get it right the first time! You will be able to get either of these programs to give you the SQL that constructs tables, or makes queries etc... That'd be enough to keep you going for a couple of weeks I guess. Also, some things make more sense in a NoSQL database, so have a look at something like MongoDB or CouchDB and how their design works differently. That's probably another couple of weeks. Also worth checking out would be http://dabodev.com hth Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Py2.7/FreeBSD: maximum number of open files
On Nov 14, 5:03 pm, Tobias Oberstein tobias.oberst...@tavendo.de wrote: I need 50k sockets + 100 files. Thus, this is even more strange: the Python (a Twisted service) will happily accept 50k sockets, but as soon as you do open() a file, it'll bail out. A limit of 32k smells like a overflow in a signed int. Perhaps your system is able and configured to handle more than 32k FDs but you hit an artificial limit because some C code or API has a overflow. This seems to be a known bug in FreeBSDhttp://lists.freebsd.org/pipermail/freebsd-bugs/2010- July/040689.html This is unbelievable. I've just tested: the bug (in libc) is still there on FreeBSD 8.2 p3 ... both on i386 _and_ amd64. Now I'm f***d;( A last chance: is it possible to compile Python for not using libc fopen(), but the Posix open()? Thanks anyway for this hint! Have you tried/or is it possible to get your 100 or whatever files first, before your sockets? hth Jon -- http://mail.python.org/mailman/listinfo/python-list
Re: Get keys from a dicionary
On Nov 11, 1:31 pm, macm moura.ma...@gmail.com wrote: Hi Folks I pass a nested dictionary to a function. def Dicty( dict[k1][k2] ): print k1 print k2 There is a fast way (trick) to get k1 and k2 as string. Whithout loop all dict. Just it! Regards macm I've tried to understand this, but can't tell if it's a question or statement, and even then can't tell what the question or statement is... Care to eloborate? -- http://mail.python.org/mailman/listinfo/python-list
Re: simple file flow question with csv.reader
On Nov 2, 11:50 pm, Terry Reedy tjre...@udel.edu wrote: On 11/2/2011 7:06 PM, Dennis Lee Bieber wrote: On Wed, 2 Nov 2011 14:13:34 -0700 (PDT), Mattmacma...@gmail.com declaimed the following in gmane.comp.python.general: I have a few hundred .csv files, and to each file, I want to manipulate the data, then save back to the original file. That is dangerous. Better to replace the file with a new one of the same name. Option 1: Read the file completely into memory (your example is reading line by line); close the reader and its file; reopen the file for wb (delete, create new); open CSV writer on that file; write the memory contents. and lose data if your system crashes or freezes during the write. Option 2: Open a temporary file wb; open a CSV writer on the file; for each line from the reader, update the data, send to the writer; at end of reader, close reader and file; delete original file; rename temporary file to the original name. This works best if new file is given a name related to the original name, in case rename fails. Alternative is to rename original x to x.bak, write or rename new file, then delete .bak file. -- Terry Jan Reedy To the OP, I agree with Terry, but will add my 2p. What is this meant to achieve? row = range(10) print ,row[0],row[4],\n,row[1], \n, , row[2], \n, row[3] 0 4 1 2 3 Is something meant to read this afterwards? I'd personally create a subdir called db, create a sqlite3 db, then load all the required fields into it (with a column for filename)... it will either work or fail, then if it succeeds, start overwriting the originals - just a select * from some_table will do, using itertools.groupby on the filename column, changing the open() request etc... just my 2p mind you, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: understand program used to create file
On Nov 1, 7:27 pm, pacopyc paco...@gmail.com wrote: Hi, I have about 1 files .doc and I want know the program used to create them: writer? word? abiword? else? I'd like develop a script python to do this. Is there a module to do it? Can you help me? Thanks My suggestion would be the same as DaveA's. This gives you the format it was *written* in. (Saved a blank OO document as 95/97/XP Word DOC under Linux) jon@forseti:~/filetest$ file * saved-by-OO.doc: CDF V2 Document, Little Endian, Os: Windows, Version 1.0, Code page: -535, Author: jon , Revision Number: 0, Create Time/ Date: Mon Oct 31 20:47:30 2011 I'd be impressed if you could discover the program that did *write* it; I'd imagine you'd need something that understood some meta-data in the format (if the format has a kind of 'created_by' field, for instance), or depend on nuisances which give away that a certain program wrote data in another's native format. Assuming the former, what might be possible: 1) Grab a magic number lookup list 2) Grab 8 (I think that should be all that's needed, but hey ummm..) bytes from the start of each file 3) Look it up in the magic number list 4) If you got something great, if not compare 7, 6, 5, 4 bytes... etc... until you get a hit or bail out (Or just find a Windows port of 'file') HTH Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Loop through a dict changing keys
On Oct 16, 12:53 am, PoD p...@internode.on.net wrote:
On Sat, 15 Oct 2011 11:00:17 -0700, Gnarlodious wrote:
What is the best way (Python 3) to loop through dict keys, examine the
string, change them if needed, and save the changes to the same dict?
So for input like this:
{'Mobile': 'string', 'context': 'malicious code', 'order': '7',
'time': 'True'}
I want to booleanize 'True', turn '7' into an integer, escape
'malicious code', and ignore 'string'.
Any elegant Python way to do this?
-- Gnarlie
How about
data = {
'Mobile': 'string',
'context': 'malicious code',
'order': '7',
'time': 'True'}
types={'Mobile':str,'context':str,'order':int,'time':bool}
for k,v in data.items():
data[k] = types[k](v)
Bit of nit-picking, but:
bool('True')
True
bool('False')
True
bool('')
False
--
http://mail.python.org/mailman/listinfo/python-list
Re: Reading a file into a data structure....
On Oct 13, 10:59 pm, MrPink tdsimp...@gmail.com wrote:
This is a continuing to a post I made in
August:http://groups.google.com/group/comp.lang.python/browse_thread/thread/...
I got some free time to work with Python again and have some followup
questions.
For example, I have a list in a text file like this:
Example list of lottery drawings:
date,wb,wb,wb,wb,wb,bb
4/1/2011,5,1,45,23,27,27
5/1/2011,15,23,8,48,22,32
6/1/2011,33,49,21,16,34,1
7/1/2011,9,3,13,22,45,41
8/1/2011,54,1,24,39,35,18
Ticket:
startdate,enddate,wb,wb,wb,wb,wb,bb
4/1/2011,8/1/2011,5,23,32,21,3,27
I am trying to determine the optimal way to organize the data
structure of the drawing list, search the drawing list, and mark the
matches in the drawing list.
f = open(C:\temp\drawinglist.txt, r)
lines = f.readlines()
f.close()
drawing = lines[1].split()
The results in drawing is this:
drawing[0] = '4/1/2011'
drawing[1] = '5'
drawing[2] = '1'
drawing[3] = '45'
drawing[4] = '23'
drawing[5] = '27'
drawing[6] = '27'
I need to convert drawing[0] to a date datatype. This works, but I'm
sure there is a better way.
from datetime import date
month, day, year = drawing[0].split('/')
drawing[0] = date(int(year), int(month), int(day))
For searching, I need to determine if the date of the drawing is
within the date range of the ticket. If yes, then mark which numbers
in the drawing match the numbers in the ticket.
ticket[0] = '4/1/2011'
ticket[0] = '8/1/2011'
ticket[0] = '5'
ticket[0] = '23'
ticket[0] = '32'
ticket[0] = '21'
ticket[0] = '3'
ticket[0] = 27'
drawing[0] = '4/1/2011' (match)
drawing[1] = '5' (match)
drawing[2] = '1'
drawing[3] = '45'
drawing[4] = '23' (match)
drawing[5] = '27'
drawing[6] = '27' (match)
I'm debating on structuring the drawing list like this:
drawing[0] = '4/1/2011'
drawing[1][0] = '5'
drawing[1][1] = '1'
drawing[1][2] = '45'
drawing[1][3] = '23'
drawing[1][4] = '27'
drawing[2] = '27'
Sort drawing[1] from low to high
drawing[1][0] = '1'
drawing[1][1] = '5'
drawing[1][2] = '23'
drawing[1][3] = '27'
drawing[1][4] = '45'
I want to keep the drawing list in memory for reuse.
Any guidance would be most helpful and appreciated.
BTW, I want to learn, so be careful not to do too much of the work for
me.
I'm using WingIDE to do my work.
Thanks,
- Use the csv module to read the file
- Use strptime to process the date field
- Use a set for draw numbers (you'd have to do pure equality on the
bb)
- Look at persisting in a sqlite3 DB (maybe with a custom convertor)
hth,
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Looking for browser emulator
On Oct 14, 3:19 am, Roy Smith r...@panix.com wrote: I've got to write some tests in python which simulate getting a page of HTML from an http server, finding a link, clicking on it, and then examining the HTML on the next page to make sure it has certain features. I can use urllib to do the basic fetching, and lxml gives me the tools to find the link I want and extract its href attribute. What's missing is dealing with turning the href into an absolute URL that I can give to urlopen(). Browsers implement all sorts of stateful logic such as if the URL has no hostname, use the same hostname as the current page. I'm talking about something where I can execute this sequence of calls: urlopen(http://foo.com:/bar;) urlopen(/baz) and have the second one know that it needs to get http://foo.com:/baz;. Does anything like that exist? I'm really trying to stay away from Selenium and go strictly with something I can run under unittest. lxml.html.make_links_absolute() ? -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for browser emulator
On Oct 14, 3:19 am, Roy Smith r...@panix.com wrote: I've got to write some tests in python which simulate getting a page of HTML from an http server, finding a link, clicking on it, and then examining the HTML on the next page to make sure it has certain features. I can use urllib to do the basic fetching, and lxml gives me the tools to find the link I want and extract its href attribute. What's missing is dealing with turning the href into an absolute URL that I can give to urlopen(). Browsers implement all sorts of stateful logic such as if the URL has no hostname, use the same hostname as the current page. I'm talking about something where I can execute this sequence of calls: urlopen(http://foo.com:/bar;) urlopen(/baz) and have the second one know that it needs to get http://foo.com:/baz;. Does anything like that exist? I'm really trying to stay away from Selenium and go strictly with something I can run under unittest. lxml.html.make_links_absolute() ? -- http://mail.python.org/mailman/listinfo/python-list
Re: Usefulness of the not in operator
On Oct 8, 11:42 am, candide cand...@free.invalid wrote:
Python provides
-- the not operator, meaning logical negation
-- the in operator, meaning membership
On the other hand, Python provides the not in operator meaning
non-membership. However, it seems we can reformulate any not in
expression using only not and in operation. For instance
'th' not in python
False
not ('th' in python)
False
So what is the usefulness of the not in operator ? Recall what Zen of
Python tells
There should be one-- and preferably only one --obvious way to do it.
You would seriously prefer the later?
Guess I'll have to start writing stuff like:
10 - 5 as 10 + -5 (as obviously the - is redundant as an operation),
and 10 / 2 as int(10 * .5) or something, who needs a divide!?
Jokely yours,
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Simplest way to resize an image-like array
On Sep 30, 5:40 pm, John Ladasky lada...@my-deja.com wrote: Hi folks, I have 500 x 500 arrays of floats, representing 2D grayscale images, that I need to resample at a lower spatial resolution, say, 120 x 120 (details to follow, if you feel they are relevant). I've got the numpy, and scipy, and matplotlib. All of these packages hint at the fact that they have the capability to resample an image- like array. But after reading the documentation for all of these packages, none of them make it straightforward, which surprises me. For example, there are several spline and interpolation methods in scipy.interpolate. They seem to return interpolator classes rather than arrays. Is there no simple method which then calls the interpolator, and builds the resampled array? Yes, I can do this myself if I must -- but over the years, I've come to learn that a lot of the code I want is already written, and that sometimes I just have to know where to look for it. Thanks! Is something like http://docs.scipy.org/doc/scipy/reference/generated/scipy.misc.imresize.html#scipy.misc.imresize any use? -- http://mail.python.org/mailman/listinfo/python-list
Re: Wrote a new library - Comments and suggestions please!
On Sep 27, 6:33 pm, Steven D'Aprano steve +comp.lang.pyt...@pearwood.info wrote: Robert Kern wrote: On 9/27/11 10:24 AM, Tal Einat wrote: I don't work with SAS so I have no reason to invest any time developing for it. Also, as far as I can tell, SAS is far from free or open-source, meaning I definitely am not interested in developing for it. I don't think he's suggesting that you drop what you are doing in Python and start working with SAS. He is suggesting that you look at the similar procedures that exist in the SAS standard library for inspiration. Yeah, inspiration on what *not* to do. I googled on SAS PROC FREQ and found this: http://support.sas.com/documentation/cdl/en/procstat/63104/HTML/defau... All the words are in English, but I have no idea what the function does, how you would call it, and what it returns. Would it have been so hard to show a couple of examples? Documentation like that really makes me appreciate the sterling work done on Python's docs. This tutorial: http://www2.sas.com/proceedings/sugi30/263-30.pdf is much clearer. -- Steven Yes - I definitely do not like the SAS docs - in fact, when I last had to buy the product, it was something like £5k for the BASE system, then if I wanted ODBC it was another £900, and the proper manuals were something stupid like another £1k (and only in hard copy) - this was a good 5/6 years ago though... (oh, and for a very basic course, it was £1.2k a day for staff to train) *sighs* [oh, and if I wanted a 'site' licence, we were talking 6 digits] Anyway, Robert Kern correctly interpreted me. I was not suggesting to the OP that he move to SAS (heaven forbid), I was indeed suggesting that he look into what similar systems have (that I have experience with and appreciate), and he acknowledges that is not present in Python, and ummm, take inspiration and quite possibly rip 'em off. A decent tabulate/cross-tabulation and statistics related there-to library is something I'd be willing to assist with and put time into. Cheers, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Wrote a new library - Comments and suggestions please!
On Sep 26, 12:23 pm, Tal Einat talei...@gmail.com wrote: The library is called RunningCalcs and is useful for running several calculations on a single iterable of values. https://bitbucket.org/taleinat/runningcalcs/http://pypi.python.org/pypi/RunningCalcs/ I'd like some input on how this could be made more useful and how to spread the word about it. The library contains the base RunningCalc class and implementations of sub-classes for common calculations: sum, min/max, average, variance standard deviation, n-largest n-smallest. Additionaly a utility function apply_in_parallel() is supplied which makes running several calculations on an iterable easy (and fast!). Straight-forward example: mean_rc, stddev_rc = RunningMean(), RunningStdDev() for x in values: mean_rc.feed(x) stddev_rc.feed(x) mean, stddev = mean_rc.value, stddev_rc.value Examples using apply_in_parallel(): mean, stddev = apply_in_parallel(values, [RunningMean(), RunningStdDev()]) five_smallest, five_largest = apply_in_parallel(values, [RunningNSmallest(5), RunningNLargest(5)]) Comments and suggestions would be highly appreciated! You may not of heard of it, but the SAS language has something called PROC FREQ... I'm imagining that maybe this is where you should be taking this. Sorry I can't comment on the code, as I haven't really got time, but have a look! (I'd be willing to invest sometime with you, if you agree that's where something like this should be going...) Cheers, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Need help with simple OOP Python question
On Sep 5, 3:43 pm, Peter Otten __pete...@web.de wrote: Kristofer Tengström wrote: Thanks everyone, moving the declaration to the class's __init__ method did the trick. Now there's just one little problem left. I'm trying to create a list that holds the parents for each instance in the hierarchy. This is what my code looks like now: - class A: def __init__(self, parents=None): self.sub = dict() if parents: You should explicitly test for None here; otherwise in a call like ancestors = [] a = A(anchestors) the list passed as an argument will not be used, which makes fore confusing behaviour. self.parents = parents else: self.parents = [] def sub_add(self, cls): hierarchy = self.parents hierarchy.append(self) Here you are adding self to the parents (that should be called ancestors) and pass it on to cls(...). Then -- because it's non-empty -- it will be used by the child, too, and you end up with a single parents list. obj = cls(hierarchy) self.sub[obj.id] = obj While the minimal fix is to pass a copy def sub_add(self, cls): obj = cls(self.parents + [self]) self.sub[obj.id] = obj I suggest that you modify your node class to keep track only of the direct parent instead of all ancestors. That makes the implementation more robust when you move a node to another parent. I may not be understanding the OP correctly, but going by what you've put here, I might be tempted to take this kind of stuff out of the class's and using a graph library (such as networkx) - that way if traversal is necessary, it might be a lot easier. But once again, I must say I'm not 100% sure what the OP wants to achieve... Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: How does this work?
On Jun 5, 4:37 am, Ben Finney ben+pyt...@benfinney.id.au wrote: jyoun...@kc.rr.com writes: I was surfing around looking for a way to split a list into equal sections. I came upon this algorithm: f = lambda x, n, acc=[]: f(x[n:], n, acc+[(x[:n])]) if x else acc f(Hallo Welt, 3) ['Hal', 'lo ', 'Wel', 't'] (http://stackoverflow.com/questions/312443/how-do-you-split-a-list-int...) This is an excellent example of why “clever” code is to be shunned. Whoever wrote this needs to spend more time trying to get their code past a peer review; the above would be rejected until it was re-written to be clear. Here is my attempt to write the above to be clear (and fixing a couple of bugs too): def split_slices(seq, slicesize, accumulator=None): Return a list of slices from `seq` each of size `slicesize`. :param seq: The sequence to split. :param slicesize: The maximum size of each slice. :param accumulator: A sequence of existing slices to which ours should be appended. :return: A list of the slices. Each item will be a slice from the original `seq` of `slicesize` length; the last item may be shorter if there were fewer than `slicesize` items remaining. if accumulator is None: accumulator = [] if seq: slice = seq[:slicesize] result = split_slices( seq[slicesize:], slicesize, accumulator + [slice]) else: result = accumulator return result It doesn't work with a huge list, but looks like it could be handy in certain circumstances. I'm trying to understand this code, but am totally lost. I know a little bit about lambda, as well as the ternary operator In Python, ‘lambda’ is merely an alternative syntax for creating function objects. The resulting object *is* a function, so I've written the above using the ‘def’ syntax for clarity. The ternary operator is often useful for very simple expressions, but quickly becomes too costly to read when the expression is complex. The above is one where the writer is so much in love with the ternary operator that they have crammed far too much complexity into a single expression. Just curious if anyone could explain how this works or maybe share a link to a website that might explain this? Does the above help? -- \ “We must find our way to a time when faith, without evidence, | `\ disgraces anyone who would claim it.” —Sam Harris, _The End of | _o__) Faith_, 2004 | Ben Finney Just my 2p, but isn't the itertools grouper recipe prudent? -- http://mail.python.org/mailman/listinfo/python-list
Re: checking if a list is empty
On May 7, 12:51 am, Ian Kelly ian.g.ke...@gmail.com wrote:
On Fri, May 6, 2011 at 4:21 PM, Philip Semanchuk phi...@semanchuk.com wrote:
What if it's not a list but a tuple or a numpy array? Often I just want to
iterate through an element's items and I don't care if it's a list, set,
etc. For instance, given this function definition --
def print_items(an_iterable):
if not an_iterable:
print The iterable is empty
else:
for item in an_iterable:
print item
I get the output I want with all of these calls:
print_items( list() )
print_items( tuple() )
print_items( set() )
print_items( numpy.array([]) )
But sadly it fails on iterators:
print_items(xrange(0))
print_items(-x for x in [])
print_items({}.iteritems())
My stab:
from itertools import chain
def print_it(iterable):
it = iter(iterable)
try:
head = next(it)
except StopIteration:
print 'Empty'
return
for el in chain( (head,), it ):
print el
Not sure if I'm truly happy with that though.
Jon
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Finding empty columns. Is there a faster way?
On Apr 21, 5:40 pm, nn prueba...@latinmail.com wrote:
time head -100 myfile /dev/null
real 0m4.57s
user 0m3.81s
sys 0m0.74s
time ./repnullsalt.py '|' myfile
0 1 Null columns:
11, 20, 21, 22, 23, 24, 25, 26, 27, 30, 31, 33, 45, 50, 68
real 1m28.94s
user 1m28.11s
sys 0m0.72s
import sys
def main():
with open(sys.argv[2],'rb') as inf:
limit = sys.argv[3] if len(sys.argv)3 else 1
dlm = sys.argv[1].encode('latin1')
nulls = [x==b'' for x in next(inf)[:-1].split(dlm)]
enum = enumerate
split = bytes.split
out = sys.stdout
prn = print
for j, r in enum(inf):
if j%100==0:
prn(j//100,end=' ')
out.flush()
if j//100=limit:
break
for i, cur in enum(split(r[:-1],dlm)):
nulls[i] |= cur==b''
print('Null columns:')
print(', '.join(str(i+1) for i,val in enumerate(nulls) if val))
if not (len(sys.argv)2):
sys.exit(Usage: +sys.argv[0]+
delimiter filename limit)
main()
What's with the aliasing enumerate and print??? And on heavy disk IO I
can hardly see that name lookups are going to be any problem at all?
And why the time stats with /dev/null ???
I'd probably go for something like:
import csv
with open('somefile') as fin:
nulls = set()
for row in csv.reader(fin, delimiter='|'):
nulls.update(idx for idx,val in enumerate(row, start=1) if not
val)
print 'nulls =', sorted(nulls)
hth
Jon
--
http://mail.python.org/mailman/listinfo/python-list
Re: TextWrangler run command not working properly
On Apr 14, 9:52 pm, Fabio oakw...@email.it wrote: Hi to all, I have troubles with TextWrangler run command in the shebang (#!) menu. I am on MacOSX 10.6.7. I have the built-in Python2.5 which comes installed by mother Apple. Then I installed Python2.6, and left 2.5 untouched (I was suggested to leave it on the system, since something might need it). I ran the Update Shell Profile.command, and now if I launch python in the terminal it happily launches the 2.6 version. Then I installed some libraries (scipy and matplotlib) on this newer 2.6 version. They work, and everything is fine. Then, I started to use TexWrangler, and I wanted to use the shebang menu, and run command. I have the #! first line pointing to the 2.6 version. It works fine, as long as I don't import the libraries, in which case it casts an error saying: ImportError: No module named scipy Maybe for some reason it points to the old 2.5 version. But I might be wrong and the problem is another... I copy here the first lines in the terminal window if i give the run in terminal command Last login: Thu Apr 14 22:38:26 on ttys000 Fabio-Mac:~ fabio$ /var/folders/BS/BSS71XvjFKiJPH3Wqtx90k+++TM/-Tmp-/Cleanup\ At\ Startup/untitled\ text-324506443.860.command ; exit; Traceback (most recent call last): File /Users/fabio/Desktop/test.py, line 3, in module import scipy as sp ImportError: No module named scipy logout [Process completed] where the source (test.py) contains just: #!/usr/bin/python2.6 import scipy as sp print hello world Any clue? Thanks Fabio http://www.velocityreviews.com/forums/t570137-textwrangler-and-new-python-version-mac.html ? -- http://mail.python.org/mailman/listinfo/python-list
Re: pattern matching
On Feb 24, 2:11 am, monkeys paw mon...@joemoney.net wrote:
if I have a string such as 'td01/12/2011/td' and i want
to reformat it as '20110112', how do i pull out the components
of the string and reformat them into a DDMM format?
I have:
import re
test = re.compile('\d\d\/')
f = open('test.html') # This file contains the html dates
for line in f:
if test.search(line):
# I need to pull the date components here
I second using an html parser to extact the content of the TD's, but I
would also go one step further reformatting and do something such as:
from time import strptime, strftime
d = '01/12/2011'
strftime('%Y%m%d', strptime(d, '%m/%d/%Y'))
'20110112'
That way you get some validation about the data, ie, if you get
'13/12/2011' you've probably got mixed data formats.
hth
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Arrays/List, filters, Pytho, Ruby
On Feb 11, 9:24 pm, LL.Snark ll.sn...@gmail.com wrote:
Hi,
I'm looking for a pythonic way to translate this short Ruby code :
t=[6,7,8,6,7,9,8,4,3,6,7]
i=t.index {|x| xt.first}
If you don't know Ruby, the second line means :
What is the index, in array t, of the first element x such that xt[0].
If can write it in python several ways :
t=[6,7,8,6,7,9,8,4,3,6,7]
i=0
while t[i]=t[0] : i+=1
... not pythonic I think...
Or :
t=[6,7,8,6,7,9,8,4,3,6,7]
i=[j for j in range(len(t)) if t[j]t[0]][0]
...too cryptic...
I'm using Python 3.
Thx
My take (but using Python 2.x):
import operator as op
from functools import partial
from itertools import islice
t = [6,7,8,6,7,9,8,4,3,6,7]
def first_conditional(seq, first=op.itemgetter(0), pred=op.gt):
f = first(seq)
cmpfunc = partial(pred, f)
for idx, val in enumerate(islice(seq, 1, None)):
if cmpfunc(val): return idx + 1
return -1 # or raise an exception?
Off top of head, so needs work, but is fairly generic.
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: For loop comprehensions
On Feb 11, 11:10 pm, Benjamin S Wolf bsw...@google.com wrote: It occurred to me as I was writing a for loop that I would like to write it in generator comprehension syntax, eg. for a in b if c: rather than using one of the more verbose but allowable syntaxes: for a in (x for x in b if c): for a in b: if not c: continue Python 3.1 does not support for comprehensions, and a few cursory searches of PEPs and this list don't turn up anything. I like the idea enough to suggest it though I'm unfamiliar with the PEP/feature request process (PEP 1 pointed me here). What do other people think? --Ben Can't help with the PEP stuff, but ummm... is anything wrong with filter()? For some things, I personally find it more readable in some circumstances than list/gen comps. I don't mind predicates at the end, but sometimes they're more obvious at the front. Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Converting getCSS Count Code from java to python
On Feb 1, 4:23 am, SMERSH009 smersh0...@gmail.com wrote:
Hi, I'd love some help converting this code to the python equivalent:
private int getCSSCount(String aCSSLocator){
String jsScript = var cssMatches = eval_css(\%s\,
window.document);cssMatches.length;;
return Integer.parseInt(selenium.getEval(String.format(jsScript,
aCSSLocator)));
}
http://www.eviltester.com/index.php/2010/03/13/a-simple-getcsscount-h...
Thanks for the help
Maybe?
http://codespeak.net/lxml/dev/cssselect.html
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: how to tell if cursor is sqlite.Cursor or psycopg2.Cursor
On Jan 24, 7:44 pm, dmaziuk dmaz...@bmrb.wisc.edu wrote: Hi everyone, I've wrapper class around some sql statements and I'm trying to add a method that does: if my_cursor is a sqlite cursor, then run select last_insert_rowid() else if it's a psycopg2 cursor, then run select currval( 'my_sequence' ) etc. The best I can come up with is import both psycopg2 and sqlite and then do if isinstance( self._curs, sqlite.Cursor ) : ... elif isinstance( self._curs, psycopg2._psycopg.cursor ) : ... and I can't help thinking there has to be another way to find out what kind of thing self._curs is. Is there a better way? TIA Dima I'm not 100% sure but maybe look at SQLAlchemy (or other Python ORMs) as a wrapper. That *might* abstract the last ID across different DB's. And still enable direct SQL querying. Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: statement level resumable exception
On Jan 21, 8:41 am, ilejn ilja.golsht...@gmail.com wrote: Arnaud, it looks like a solution. Perhaps it is better than plain try/accept and than proxy class with __getattr__. It is not for free, e.g. because syntax check such as parentheses matching is lazy too, though looks very interesting. Thanks a lot! On Jan 21, 10:41 am, Arnaud Delobelle arno...@gmail.com wrote: ilejn ilja.golsht...@gmail.com writes: Arnaud, these lists are not generated. Actually these lists are a sort of interpreted programs and contain some application logic. Here is an example [ [PUSH, [get_modified_interface, req]], [TIMEOUT, 3], [PULL, [out_interface, '']], [PULL, [err_interface, '']], [PULL, [out_mined_interface, req]], ] If any interface name is unknown the list must not be invoked (in other words, f function call with this list must be somehow bypassed). Thanks. You could still use the same idea and delay evaluation of the lists. E.g. prg1 = [ [PUSH, [get_modified_interface, req]], [TIMEOUT, 3], ... prg2 = [ [OPCODE, [arguments, blah]], ... ... prgN = ... for prg in prg1, prg2, ..., prgN: try: prg = eval(prg) except NameError: continue f(prg) -- Arnaud Best regards, Ilja Golshtein Not sure if a good idea or not, but: I would probably use pyparsing and create a small grammar to parse your list data. If parsing an entry with an unknown interface, then skip to the next list entry. If the entire list parses, then you can execute your function calls. hth Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Developing a program to make a family tree.
On Jan 14, 7:39 pm, Ata Jafari a.j.romani...@gmail.com wrote: Hi there. I'm trying to develop a program like family tree maker. I have all information, so there is no need to search on the net. This must be something like trees. Can someone help me? I'm at the beginning. Thanks. -- Ata J. Tabrizi atae.tabr...@metu.edu.tr If you're after mature and actively developed Genealogy software developed in Python, then check out http://gramps-project.org/ The developer list is very friendly. Otherwise, you're in for a struggle, as you need to choose a storage back-end, a GUI (wxWindows/GTK/Qt4 etc...), how to handle GEDCOM format (unless it's not going to be compatible with other software), does it need to produce web pages/reports (and in what formats). I strongly suggest looking at GRAMPS and see what you're setting yourself up for :) hth Jon -- http://mail.python.org/mailman/listinfo/python-list
Re: Career path - where next?
On Jan 12, 4:37 pm, Alan Harris-Reid aharrisr...@googlemail.com wrote: Hi there, I wonder if any Python folk out there can help me. For many years I was a contractor developing desktop and web applications using Visual Foxpro as my main language, with Foxpro, SQL-server and Oracle as back-end databases. Unfortunately Foxpro was killed-off by Microsoft, hence my work dried-up and my last 'big' contract ended about a year ago. Since then I have taken time off programming doing house-renovation, and in the last 6 months I have been updating my programming skills by learning Python (3) with SQLite, JavaScript, HTML and CSS to a level where I can create and deploy data-based web-sites. My situation now is that I am reasonably comfortable with the above languages and am now in a position where I wish to return to employment using my new and/or existing skills (contract/permanent, full/part-time or teleworking). However, I have yet to find any UK vacancy which will accept a relative 'beginner' - they all require at least 2-3 years Python in a commercial environment. It's a catch-22 situation - it's hard to get a job without experience, but you need a job to get experience in the 1st place! I would even consider doing small projects for nothing so that I can 'get my foot in the door' (although I hope to be wise-enough to know when I am being taken advantage of!). I am also mailing CVs to agencies I think may be interested. If anyone out has ideas as to how to proceed towards achieving my goal, I would be grateful for any advice. Regards, Alan Harris-Reid Hi Alan, Just some ideas (not in any order, just as they're thought of):- - Emphasise your experience with Oracle SQL Server, and use Python as a I also have. It may be someone will accept that as viable (saves them a DBA or something), and maybe you'll get into an experienced group and get on the job training. (I assume you have good SQL skills). - Look at cwjobs.co.uk / monster / etc..., and search for Python. Get a list of agencies there. Phone them *first*, explain what is it you've done, and what you can do. If the person seems to know what they're talking about send your CV - but chase often. - Look at web-frameworks. Django seems to be the most listed for required/nice to have. Also check out javascript-frameworks - jquery extjs are the biggest 2, so at least you can say you've had some experience. - Perhaps phone your local job centre, and ask for a contact for their local volunteer centre. They might have something like work for a small charity that just needs a couple of pages done. The idea being: 1) If it's a cause you believe in, it makes up for not getting paid; 2) You can use it as an example and reference; 3) You might be able to use it as networking - might get a free lunch from an event and meet someone with money, that's impressed with your good will and work, and just happens to have a job going spare... - Build a copy of your CV designed for the web. Make sure it looks good, is HTML/CSS compliant, and even add some nice interactive stuff to it, and include it as a link in your CV. [The other thing you can do, is only display the CV on entry of a short PIN (different for each one you send - '2431' or something'), then you can log who's bothered looking at it, and when, enabling timing of a follow-up better)]. - Look in local papers for local companies that offer not necessarily web design, but possibly just print design. See if you can't have a chat with them and get some work your way. Other options might be new- starts up, non-chain pubs, community/sports clubs, a local church for fund-raising, your local chinese/indian takeaway - wouldn't hurt to put their menu online with an online order form would it!? [What you might find about this, is that as they're not likely to be technical, you can take your own time, charge a reasonable amount, experiment a little and learn, and not have too tight deadlines or someone looking over your shoulder]. Brain (or somewhere else) dump finished. hth Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Building sys.path at run-time?
On Dec 30, 4:24 am, Roy Smith r...@panix.com wrote: In article 87k4irhpoa@benfinney.id.au, Ben Finney ben+pyt...@benfinney.id.au wrote: Roy Smith r...@panix.com writes: I've got a problem that I'm sure many people have solved many times. Our project has a bunch of python scripts A very common problem. The solution is to switch to Perl. (Merry solstice silliness, everyone :-) I have another problem. I hit the Post button by accident. Please ignore. Sorry all, festive joy and all that. +1 for a laugh that your problem is [a] bunch of python scripts, and another +1 'cos I have another problem... Damn it, if this was 10-15 years ago, with bash.org/qdb.us etc..., I'd have posted that... Thanks Roy for the laugh, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Hosting a Python based TCP server
On Dec 23, 12:01 pm, Oltmans rolf.oltm...@gmail.com wrote: Hi all, I'm writing a very small TCP server(written in Python) and now I want to host it on some ISP so that it can be accessed anywhere from the Internet. I've never done that before so I thought I should ask for some advice. Do you guys know any good ISP that can let me do that? Most importantly, what is usually involved in order to make this happen? Please pardon my ignorance and I will really appreciate your reply. Thanks in advance. Check out http://wiki.python.org/moin/PythonHosting There's quite a few and they vary in features/pricing/OS used etc... but a lot of the info and links are there for you to find one right for you. hth Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Partition Recursive
On Dec 23, 5:26 pm, macm moura.ma...@gmail.com wrote: Hi Folks I have this: url = 'http://docs.python.org/dev/library/stdtypes.html? highlight=partition#str.partition' So I want convert to myList = ['http',':','//','docs','.','python','.','org','/','dev','/','library','/','stdtypes','.','html','?','highlight','=','partition','#','str','.','partition'] The reserved char are: specialMeaning = [//,;,/, ?, :, @, = , ,#] Regards Mario I would use urlparse.urlsplit, then split further, if required. urlsplit(url) SplitResult(scheme='http', netloc='docs.python.org', path='/dev/ library/stdtypes.html', query='highlight=partition', fragment='str.partition') Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: help with link parsing?
On Dec 22, 4:24 pm, Colin J. Williams cjwilliam...@gmail.com
wrote:
On 21-Dec-10 12:22 PM, Jon Clements wrote:
import lxml
from urlparse import urlsplit
doc = lxml.html.parse('http://www.google.com')
print map(urlsplit, doc.xpath('//a/@href'))
[SplitResult(scheme='http', netloc='www.google.co.uk', path='/imghp',
query='hl=entab=wi', fragment=''), SplitResult(scheme='http',
netloc='video.google.co.uk', path='/', query='hl=entab=wv',
fragment=''), SplitResult(scheme='http', netloc='maps.google.co.uk',
path='/maps', query='hl=entab=wl', fragment=''),
SplitResult(scheme='http', netloc='news.google.co.uk', path='/nwshp',
query='hl=entab=wn', fragment=''), ...]
Jon,
What version of Python was used to run this?
Colin W.
2.6.5 - the lxml library is not a standard module though and needs to
be installed.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Modifying an existing excel spreadsheet
On Dec 20, 9:56 pm, Ed Keith e_...@yahoo.com wrote: I have a user supplied 'template' Excel spreadsheet. I need to create a new excel spreadsheet based on the supplied template, with data filled in. I found the tools herehttp://www.python-excel.org/, andhttp://sourceforge.net/projects/pyexcelerator/. I have been trying to use the former, since the latter seems to be devoid of documentation (not even any docstrings). My first thought was to copy the template, open the copy, modify it and save the modifications. But it looks like if I open an existing spreadsheet it must be read only. So I tried to open the template, copy it to a new spreadsheet and write the new spreadsheet, but I can't seem to copy the images, and it looks like copying the formatting is going to be difficult. Can anyone give me any tips or advice? Thanks in advance, -EdK Ed Keith e_...@yahoo.com Blog: edkeith.blogspot.com Have you tried: http://groups.google.com/group/python-excel and searching the archives for template? Similar questions have come up before there. hth Jon -- http://mail.python.org/mailman/listinfo/python-list
Re: help with link parsing?
On Dec 20, 7:14 pm, Littlefield, Tyler ty...@tysdomain.com wrote:
Hello all,
I have a question. I guess this worked pre 2.6; I don't remember the
last time I used it, but it was a while ago, and now it's failing.
Anyone mind looking at it and telling me what's going wrong? Also, is
there a quick way to match on a certain site? like links from google.com
and only output those?
#!/usr/bin/env python
#This program is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published
#by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
#This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
#
#You should have received a copy of the GNU General Public License along
with this program. If not, see
#http://www.gnu.org/licenses/.
This script will parse out all the links in an html document and write
them to a textfile.
import sys,optparse
import htmllib,formatter
#program class declarations:
class Links(htmllib.HTMLParser):
def __init__(self,formatter):
htmllib.HTMLParser.__init__(self, formatter)
self.links=[]
def start_a(self, attrs):
if (len(attrs)0):
for a in attrs:
if a[0]==href:
self.links.append(a[1])
print a[1]
break
def main(argv):
if (len(argv)!=3):
print(Error:\n+argv[0]+ input output.\nParses input
for all links and saves them to output.)
return 1
lcount=0
format=formatter.NullFormatter()
html=Links(format)
print Retrieving data:
page=open(argv[1],r)
print Feeding data to parser:
html.feed(page.read())
page.close()
print Writing links:
output=open(argv[2],w)
for i in (html.links):
output.write(i+\n)
lcount+=1
output.close()
print(Wrote +str(lcount)+ links to +argv[2]+.);
print(done.)
if (__name__ == __main__):
#we call the main function passing a list of args, and exit with
the return code passed back.
sys.exit(main(sys.argv))
--
Thanks,
Ty
This doesn't answer your original question, but excluding the command
line handling, how's this do you?:
import lxml
from urlparse import urlsplit
doc = lxml.html.parse('http://www.google.com')
print map(urlsplit, doc.xpath('//a/@href'))
[SplitResult(scheme='http', netloc='www.google.co.uk', path='/imghp',
query='hl=entab=wi', fragment=''), SplitResult(scheme='http',
netloc='video.google.co.uk', path='/', query='hl=entab=wv',
fragment=''), SplitResult(scheme='http', netloc='maps.google.co.uk',
path='/maps', query='hl=entab=wl', fragment=''),
SplitResult(scheme='http', netloc='news.google.co.uk', path='/nwshp',
query='hl=entab=wn', fragment=''), ...]
Much nicer IMHO, plus the lxml.html has iterlinks() and other
convenience functions for handling HTML.
hth
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Scanning directories for new files?
On Dec 21, 7:17 pm, Matty Sarro msa...@gmail.com wrote: Hey everyone. I'm in the midst of writing a parser to clean up incoming files, remove extra data that isn't needed, normalize some values, etc. The base files will be uploaded via FTP. How does one go about scanning a directory for new files? For now we're looking to run it as a cron job but eventually would like to move away from that into making it a service running in the background. Not a direct answer, but I would choose the approach of letting the FTP server know when a new file has been added. For instance: http://www.pureftpd.org/project/pure-ftpd - Any external shell script can be called after a successful upload. Virus scanners and database archiveal can easily be set up. Of course, there's loads more servers, that I'm sure will have callback events or similar. Although, yes, the monitoring the file system is completely possible. hth Jon. -- http://mail.python.org/mailman/listinfo/python-list
Specialisation / Interests
Hi all, Was thinking tonight (now this morning my time): What would we consider the long time posters on c.l.p consider what they respond to and offer serious advice on. For instance: - Raymond Hettinger for algo's in collections and itertools - MRAB for regex's (never seen him duck a post where re was (not) required. - the effbot for PIL ElementTree - Mark Hammond for work on win32 - Mark Dickinson for floating point/number theory etc... Then so many others!... I'm leaving a huge amount out, so no rudeness intended - but what you think guys and gals? Cheers, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Case Sensitive Section names configparser
On Dec 8, 10:32 am, RedBaron dheeraj.gup...@gmail.com wrote: Is there any way by which configParser's get() function can be made case insensitive? I would probably subclass dict to create a string specific, case insensitive version, and supply it as the dict_type. See http://docs.python.org/library/configparser.html#ConfigParser.RawConfigParser That way your values would remain cased correctly, but lookups would be insensitive. hth Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Login using usrllib2
On Dec 1, 10:16 am, Chris Rebert c...@rebertia.com wrote:
On Wed, Dec 1, 2010 at 1:53 AM, dudeja.ra...@gmail.com wrote:
Hi All,
I'm using urllib2 module to login to an https server. However I'm unable to
login as the password is not getting accepted.
Here is the code:
import urllib2, urllib
values={'Username': 'admin', 'Password': 'admin123'}
url='https://172.25.17.20:9443'
data = urllib.urlencode(values)
data = urllib.urlencode(values)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
the_page = response.read()
print the_page
The error message I get in the_page output is the same as I get when I
access this site using a browser and supply a wrong password. It appears
password is not accepting.
I'm new to web programming. The aim is to automate user clicks on a web
server which can be replicated using python and http. Please suggest / help.
You should probably use something like Firefox's Live HTTP Headers
extension to see what exactly the web browser is doing when you
normally login to the site, so that you can correctly replicate the
browser's actions in your script.
The site may be using HTTP Basic or HTTP Digest authentication, in
which case you'll need to use urllib2.HTTPBasicAuthHandler or
urllib2.HTTPDigestAuthHandler.
Cheers,
Chris
--http://blog.rebertia.com
If the OP is doing this a lot I would suggest installing FF's Firebug
extension. Then use the Net tab to monitor activity. Also the OP might
want to look at the urllib2.HTTPSHandler.
Jon
--
http://mail.python.org/mailman/listinfo/python-list
Re: Packages at Python.org
On Dec 1, 8:56 pm, kirby.ur...@gmail.com kirby.ur...@gmail.com wrote: http://packages.python.org/dbf/ So how *do* you get source code from such a web place? I'm not finding a tar ball or installer. Sorry if I'm missing something obvious, like a link to Sourceforge. Thanks to very quick replies with pointers to http://pypi.python.org/pypi/dbf/ instead of to packages.python.org/dbf [snip] Kirby I've only had a quick play with it, however, you may want to take a look at Dabo on http://dabodev.com/ Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Packages at Python.org
On Dec 1, 10:32 pm, Ethan Furman et...@stoneleaf.us wrote: kirby.ur...@gmail.com wrote: With Microsoft abandoning Visual FoxPro come 2015, we have 100K developers jumping ship (rough guess), perhaps to dot NET, but not necessarily.** This page is potentially getting a lot of hits (I'm not privy to the analytics): http://packages.python.org/dbf/ The dbf package does not yet support index files. Normal indexes won't be too hard to add in, but I have been unable to find the algorithms used to create/work with compact index files. Does anybody know where I might find those? ~Ethan~ (author of said package) I think John Machin has done work on this, or at least mentioned it in a post somewhere. I could be wrong though - apologies to both you and John, if I'm imagining things. Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing markup.
On Nov 26, 4:03 am, MRAB pyt...@mrabarnett.plus.com wrote:
On 26/11/2010 03:28, Joe Goldthwaite wrote:
I’m attempting to parse some basic tagged markup. The output of the
TinyMCE editor returns a string that looks something like this;
pThis is a paragraph with bbold/b and iitalic/i elements in
it/ppIt can be made up of multiple lines separated by pagagraph
tags./p
I’m trying to render the paragraph into a bit mapped image. I need
to parse it out into the various paragraph and bold/italic pieces.
I’m not sure the best way to approach it. Elementree and lxml seem
to want a full formatted page, not a small segment like this one.
When I tried to feed a line similar to the above to lxml I got an
error; “XMLSyntaxError: Extra content at the end of the document”.
lxml works fine for me - have you tried:
from lxml import html
text = pThis is a paragraph with bbold/b and iitalic/i
elements in it/ppIt can be made up of multiple lines separated by
pagagraph tags./p
tree = html.fromstring(text)
print tree.findall('p')
# should print [Element p at 2b7b458, Element p at 2b7b3e8]
hth
Jon
--
http://mail.python.org/mailman/listinfo/python-list
Re: memory management - avoid swapping/paging
On 21 Oct, 16:45, Nobody nob...@nowhere.com wrote: On Thu, 21 Oct 2010 02:34:15 -0700, Jon Clements wrote: I'm after something that says: I want 512mb of physical RAM, I don't want you to page/swap it, if you can't do that, don't bother at all. Now I'm guessing, that an OS might be able to grant that, but later on have to kill the process as other higher-priority processes need RAM The mlock() system call locks a region of virtual memory into physical memory. AFAICT, Python doesn't provide an interface, but you can use ctypes for that. Thank you and Alain for this. In Linux 2.6.9 and later, the ability to lock memory is controlled via resource limits (see man 2 setrlimit, help ulimit and man 5 limits.conf). Earlier versions limit memory locking to processes owned by root or with the CAP_IPC_LOCK capability. The man and help ref's are much appreciated. It's narrowed my search space. Also, locking a specific region of memory won't necessarily help if the program code and stack are subject to paging/swapping. You can use mlockall() to lock all memory for a process. Good point. Thanks for pointers, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading Outlook .msg file using Python
On 20 Oct, 18:13, John Henry john106he...@hotmail.com wrote:
On Oct 20, 9:01 am, John Henry john106he...@hotmail.com wrote:
On Oct 20, 1:41 am, Tim Golden m...@timgolden.me.uk wrote:
On 19/10/2010 22:48, John Henry wrote:
Looks like this flag is valid only if you are getting messages
directly from Outlook. When reading the msg file, the flag is
invalid.
Same issue when accessing attachments. In addition, the MAPITable
method does not seem to work at all when trying to get attachments out
of the msg file (works when dealing with message in an Outlook
mailbox). Eitherway, the display_name doesn't work when trying to
display the filename of the attachment.
I was able to get the date by using the PR_TRANSPORT_MESSAGE_HEADERS
mapitags
Ah, thanks. As you will have realised, my code is basically geared
to reading an Outlook/Exchange message box. I hadn't really tried
it on individual message files, except my original excerpt. If it
were opportune, I'd be interested in seeing your working code.
TJG
When (and if) I finally figure out how to get it done, I surely will
make the code available. It's pretty close. All I need is to figure
out how to extract the attachments.
Too bad I don't know (and don't have) C#. This guy did it so cleanly:
http://www.codeproject.com/KB/office/reading_an_outlook_msg.aspx?msg=...
May be somebody that knows both C# and Python can convert the code
(not much code) and then the Python community will have it. As it
stands, it seems the solution is available in Java, C#, VB but
not Python.
BTW: For the benefit of future search on this topic, with the code
listed above where:
storage_flags = STGM_DIRECT | STGM_READ | STGM_SHARE_EXCLUSIVE
I had to change it to:
storage_flags = STGM_DIRECT | STGM_READ | STGM_SHARE_DENY_NONE |
STGM_TRANSACTED
otherwise I get a sharing violation
(seehttp://efreedom.com/Question/1-1086814/Opening-OLE-Compound-Documents...).
For now, I am using a brute force method (http://mail.python.org/
pipermail/python-win32/2009-February/008825.html) to get the names of
the attachments and if I need to extract the attachments, I pop up the
message in Outlook and let Outlook extract the files. Ugly but fits
my client's need for now. Hopefully there will be a cleaner solution
down the road.
Here's my code for brute forcing attachments out of the msg file (very
ugly):
def get_attachments(self, fileID):
#from win32com.storagecon import *
from win32com import storagecon
import pythoncom
flags = storagecon.STGM_READ |
storagecon.STGM_SHARE_DENY_NONE |
storagecon.STGM_TRANSACTED
try:
storage = pythoncom.StgOpenStorage (fileID, None,
flags)
except:
return []
flags = storagecon.STGM_READ | storagecon.STGM_SHARE_EXCLUSIVE
attachments=[]
for data in storage.EnumElements ():
print data[0], data[1]
if data[1] == 2 or data[0] == __substg1.0_007D001F:
stream = storage.OpenStream (data[0], None,
flags)
try:
msg = stream.Read (data[2])
except:
pass
else:
msg = repr (msg).replace(\
\x00,).strip(').replace(%23,#)
if data[0] == __substg1.0_007D001F:
try:
attachments.append(msg.split(name=\)[1].split(\)[0])
except:
pass
return attachments
Only just noticed this thread, and had something similar. I took the
following approach:-
(I'm thinking this might be relevant as you mentioned checking whether
your client's Outlook could export .EML directly, which indicates (to
me at least) that you have some control over that...)
- Set up an IMAP email server on a machine (in this case linux and
dovecot)
- Got client to set up a new account in Outlook for the new server
- Got client to use the Outlook interface to copy relevant emails (or
the whole lot) to new server
- Used the standard imaplib and related modules to do what was needed
From my POV I didn't have to mess around with proprietary formats or
deal with files. From the client's POV, they were able to, with an
interface familiar to them, add/remove what needed processing. It also
enabled multiple people at the client's site to contribute their
emails that might have been relevant for the task.
The program created a sub-folder under
memory management - avoid swapping/paging
Hi all, Is there a cross-platform way using Python to guarantee that an object will never be swapped/paged to disk? I'll be honest and say I'm really not sure if this is a particular language question or rather specific to an OS. Under linux it appears I could create a ramfs and mmap a file under that. Is there a way to do the above with mmap? I'm after something that says: I want 512mb of physical RAM, I don't want you to page/swap it, if you can't do that, don't bother at all. Now I'm guessing, that an OS might be able to grant that, but later on have to kill the process as other higher-priority processes need RAM -- that's fine. Cheers, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 249 (database api) -- executemany() with iterable?
On 12 Oct, 20:21, J. Gerlach gerlach_jo...@web.de wrote: Am 12.10.2010 17:10, schrieb Roy Smith: [A]re there any plans to update the api to allow an iterable instead of a sequence? sqlite3 (standard library, python 2.6.6., Windows 32Bit) does that already:: import sqlite3 as sql connection = sql.connect(:memory:) cursor = connection.execute( CREATE TABLE test ( id INTEGER PRIMARY KEY AUTOINCREMENT, text TEXT) ;) connection.commit() cursor.executemany( INSERT INTO test (text) VALUES ( ? ); , # A generator expression - delivers one row at a time ( (hello nr %03d! % i,) for i in xrange(100))) connection.commit() cursor.execute(SELECT * FROM test) for id_, text in cursor.fetchall(): print text, id_ What happens if you do itertools.repeat(0) instead of xrange(100) ? -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 249 (database api) -- executemany() with iterable?
On 12 Oct, 16:10, Roy Smith r...@panix.com wrote: PEP 249 says about executemany(): Prepare a database operation (query or command) and then execute it against all parameter sequences or mappings found in the sequence seq_of_parameters. are there any plans to update the api to allow an iterable instead of a sequence? I'm not understanding (probably me). Example? -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 249 (database api) -- executemany() with iterable?
On 12 Oct, 18:32, Roy Smith r...@panix.com wrote:
On Oct 12, 1:20 pm, Jon Clements jon...@googlemail.com wrote:
On 12 Oct, 16:10, Roy Smith r...@panix.com wrote:
PEP 249 says about executemany():
Prepare a database operation (query or command) and then
execute it against all parameter sequences or mappings
found in the sequence seq_of_parameters.
are there any plans to update the api to allow an iterable instead of
a sequence?
I'm not understanding (probably me). Example?
I have a dictionary, d, which has a million items in it, and want to
do something like:
executemany(update foo set bar = %s where id = %s,
d.iteritems())
If executemany accepted an iterable, that would work. But, it only
accepts a sequence, so I need to do:
executemany(update foo set bar = %s where id = %s, d.items())
which generates a million-item temporary list. Or am I mis-
understanding the PEP?
Interesting, but here's my guess...
Replace d.items() with itertools.repeat( ('a', 'b') )
So, if you have a sequence, which has a length and known size, at
least you can have an attempt at the DB operations: whether the
transaction fails or not is another thing...In short, a sequence is
finite, while an iterable may be infinite.
That's just my guess and makes sense to me at least!
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: PEP 249 (database api) -- executemany() with iterable?
On 12 Oct, 18:53, Jon Clements jon...@googlemail.com wrote:
On 12 Oct, 18:32, Roy Smith r...@panix.com wrote:
On Oct 12, 1:20 pm, Jon Clements jon...@googlemail.com wrote:
On 12 Oct, 16:10, Roy Smith r...@panix.com wrote:
PEP 249 says about executemany():
Prepare a database operation (query or command) and then
execute it against all parameter sequences or mappings
found in the sequence seq_of_parameters.
are there any plans to update the api to allow an iterable instead of
a sequence?
I'm not understanding (probably me). Example?
I have a dictionary, d, which has a million items in it, and want to
do something like:
executemany(update foo set bar = %s where id = %s,
d.iteritems())
If executemany accepted an iterable, that would work. But, it only
accepts a sequence, so I need to do:
executemany(update foo set bar = %s where id = %s, d.items())
which generates a million-item temporary list. Or am I mis-
understanding the PEP?
Interesting, but here's my guess...
Replace d.items() with itertools.repeat( ('a', 'b') )
So, if you have a sequence, which has a length and known size, at
least you can have an attempt at the DB operations: whether the
transaction fails or not is another thing...In short, a sequence is
finite, while an iterable may be infinite.
That's just my guess and makes sense to me at least!
Jon.
Actually, thinking about it some more, I would take the following
approach:
(this is only loosely do with the Python DB API mind you...)
1) Start a transaction
2) Create a temporary table
3) Bulk insert your million update records to the temp table (from my
understanding of the PEP, executemany(), is allowed to repeatedly call
execute() unless it can do something cleverer)
4) Execute an update with a from statement joining your main table and
temp table (pretty sure that's ANSI standard, and DB's should support
it -- embedded one's may not though, but if you're dealing with 1mil
records, I'm taking a guess you're not dealing with embedded)
5) End the transaction
Far more efficient as a repeated execute of 'update' will only just
hit the DB once, while an update statement with a from should allow
the DB a chance to optimise it.
Hope that makes sense, lemme know.
Cheers,
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
[issue9944] Typo in doc for itertools recipe of consume
New submission from Jon Clements jon...@googlemail.com: Very low priority. def consume(iterator, n): Advance the iterator n-steps ahead. If n is none, consume entirely. # Use functions that consume iterators at C speed. if n is None: # feed the entire iterator into a zero-length deque collections.deque(iterator, maxlen=0) else: # advance to the emtpy slice starting at position n next(islice(iterator, n, n), None) Hardly a show stoppper, and not me worth submitting a patch, but emtpy should be empty. Just thought I'd make note of it before I forgot. -- assignee: d...@python components: Documentation messages: 117339 nosy: d...@python, joncle priority: normal severity: normal status: open title: Typo in doc for itertools recipe of consume versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9944 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
re.sub: escaping capture group followed by numeric(s)
Hi All, (I reckon this is probably a question for MRAB and is not really Python specific, but anyhow...) Absolutely basic example: re.sub(r'(\d+)', r'\1', 'string1') I've been searching around and I'm sure it'll be obvious when it's pointed out, but how do I use the above to replace 1 with 11? Obviously I can't use r'\11' because there is no group 11. I know I can use a function to do it, but it seems to me there must be a way without. Can I escape r'\11' somehow so that it's group 1 with a '1' after it (not group 11). Cheers, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: re.sub: escaping capture group followed by numeric(s)
On 17 Sep, 19:59, Peter Otten __pete...@web.de wrote: Jon Clements wrote: (I reckon this is probably a question for MRAB and is not really Python specific, but anyhow...) Absolutely basic example: re.sub(r'(\d+)', r'\1', 'string1') I've been searching around and I'm sure it'll be obvious when it's pointed out, but how do I use the above to replace 1 with 11? Obviously I can't use r'\11' because there is no group 11. I know I can use a function to do it, but it seems to me there must be a way without. Can I escape r'\11' somehow so that it's group 1 with a '1' after it (not group 11). Quoting http://docs.python.org/library/re.html#re.sub In addition to character escapes and backreferences as described above, \gname will use the substring matched by the group named name, as defined by the (?Pname...) syntax. \gnumber uses the corresponding group number; \g2 is therefore equivalent to \2, but isn’t ambiguous in a replacement such as \g20. \20 would be interpreted as a reference to group 20, not a reference to group 2 followed by the literal character '0'. The backreference \g0 substitutes in the entire substring matched by the RE. Peter Thanks Peter and MRAB. I must have been through the docs half a dozen times and missed that - what a muppet! One of those days I guess... Cheers, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: palindrome iteration
On Aug 28, 11:55 am, Steven D'Aprano st...@remove-this-
cybersource.com.au wrote:
On Sat, 28 Aug 2010 09:22:13 +0300, Jussi Piitulainen wrote:
Terry Reedy writes:
On 8/27/2010 3:43 PM, Jussi Piitulainen wrote:
Dave Angel writes:
[snip]
Not everything needs to be a built-in method. There is already a standard
way to spell reverse a string:
astring[::-1]
If you don't like that, you can do this:
''.join(reversed(astring))
I've had to debug code that assumed str(reversed('abc')) == 'cba'
str(reversed('abc'))
'reversed object at 0xa66f78c'
So, a str doesn't construct like tuple/list...it's a call to
__str__().
It's designated as a friendly print out (that's my phrasing).
list('abc')
['a', 'b', 'c']
I s'pose str is special (2.6) in some way, but it doesn't parallel the
other builtins.
[Not at Terry / Steve intended -- just most relevant post to respond
to]
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
