sys.maxint in Python 2.6.1 (amd64) on Windows XP x64
Hi, I installed the amd64 version of Python 2.6.1 on my Windows XP x64 system. I was expecting sys.maxint to be 9223372036854775807 (or 2 ^63 -1), but instead I got 2147483647 (i.e., 2^31-1) just like what I got from a 32-bit version of Python. Is this by design or does it indicate a bug or an installation problem? Thank you very much! --lin -- http://mail.python.org/mailman/listinfo/python-list
Re: sys.maxint in Python 2.6.1 (amd64) on Windows XP x64
> > > I installed the amd64 version of Python 2.6.1 on my Windows XP x64 > > system. I was expecting sys.maxint to be 9223372036854775807 (or 2 ^63 > > -1), but instead I got 2147483647 (i.e., 2^31-1) just like what I got > > from a 32-bit version of Python. Is this by design or does it indicate > > a bug or an installation problem? Thank you very much! > > This is by design. In their infinitive wisdom Microsoft has decided to > make the 'long' C type always a 32 bit signed integer - even on 64bit > systems. On most Unix systems a long is at least 32 bit but usually > sizeof(ptr). > Ah, this makes sense. Thanks.. The main reason I'm trying 64-bit Python is that I want to write files bigger than 4GB. This should work on Windows x64, right? (i.e., are the pointers bona fide 64 bit?) -lin -- http://mail.python.org/mailman/listinfo/python-list
Embedding Python27 in C++ on Windows: CRT compatibility issues with VS2010?
Hello, I'm a relative python newbie but I've been tasked to figure out how to embed calls to a python library in an Excel XLL add-in. The Python/C API for doing this seems pretty straightforward, but I seem to have read somewhere online that it's important that the C++ program or DLL linking to and embedding Python must be using the same CRT as what the Python implementation dll is using. Is this true, or is the Python API written in such a way that there is no dependency on a common CRT? If there is a dependency, does that mean that I cannot use VS2010 to develop this XLL, but should use VS2008 instead, or are there other workarounds? Thanks for the help, Stephen -- http://mail.python.org/mailman/listinfo/python-list
pairwise combination of two lists
Hi Python users,
I have two lists:
li1 = ['a', 'b']
li2 = ['1', '2']
and I wish to obtain a list like this
li3 = ['a1', 'a2', 'b1', 'b2']
Is there a handy and efficient function to do this, especially when li1 and li2
are long lists.
I found zip() but it only gives [('a', '1'), ('b', '2')], not exactly what I
am looking for.
Thank you.
- Yingjie
--
http://mail.python.org/mailman/listinfo/python-list
HTML client sctript
Hi Python users, I am maintaining a website written with Python CGI scripts. To make sure the website is working well, I would like to have a script which automatically "uses" this website and checks it's output everyday. It would be better if this script runs from the clients' side. Could any one suggest any Python modules, articles, tutorials, ect. that might be helpful? Thank you. - Yingjie -- http://mail.python.org/mailman/listinfo/python-list
try... except with unknown error types
Hi Python users, I have been using try...except statements in the situations where I can expect a certain type of errors might occur. But sometimes I don't exactly know the possible error types, or sometimes I just can't "spell" the error types correctly. For example, try: response = urlopen(urljoin(uri1, uri2)) except urllib2.HTTPError: print "URL does not exist!" Though "urllib2.HTTPError" is the error type reported by Python, Python doesn't recognize it as an error type name. I tried using "HTTPError" alone too, but that's not recognized either. Does anyone know what error type I should put after the except statement? or even better: is there a way not to specify the error types? Thank you. - Yingjie -- http://mail.python.org/mailman/listinfo/python-list
Re: HTML client sctript
Hi John and Chris, Thanks for the help. I looked at both and think that mechanize suits my needs better, since it simply needs a python script and doesn't depend on Firefox. Yingjie > From: John Gordon > Mechanize seems like what you want. It's built on top of > urllib2.http://wwwsearch.sourceforge.net/mechanize/ > > From: Chris Rebert > Selenium:http://seleniumhq.org/ > -- http://mail.python.org/mailman/listinfo/python-list
Re: try... except with unknown error types
Hi Zero,
I see! This is very helpful. Thank you.
- Yingjie
On Aug 19, 2011, at 3:30 PM, Zero Piraeus wrote:
> :
>
> On 19 August 2011 15:09, Yingjie Lin wrote:
>>
>> I have been using try...except statements in the situations where I can
>> expect a certain type of errors might occur.
>> But sometimes I don't exactly know the possible error types, or sometimes I
>> just can't "spell" the error types correctly.
>> For example,
>>
>> try:
>>response = urlopen(urljoin(uri1, uri2))
>> except urllib2.HTTPError:
>>print "URL does not exist!"
>>
>> Though "urllib2.HTTPError" is the error type reported by Python, Python
>> doesn't recognize it as an error type name.
>> I tried using "HTTPError" alone too, but that's not recognized either.
>>
>> Does anyone know what error type I should put after the except statement? or
>> even better: is there a way not to specify
>> the error types? Thank you.
>
> You should always specify the error type, so that your error-handling
> code won't attempt to handle an error it didn't anticipate and cause
> even more problems.
>
> In this case, I think it's just that you haven't imported HTTPError
> into your namespace - if you do, it works:
>
>>>> from urllib2 import urlopen, HTTPError
>>>> try:
> ... response = urlopen("http://google.com/invalid";)
> ... except HTTPError:
> ... print "URL does not exist!"
> ...
> URL does not exist!
>>>>
>
> Alternatively:
>
>>>> import urllib2
>>>> try:
> ... response = urllib2.urlopen("http://google.com/invalid";)
> ... except urllib2.HTTPError:
> ... print "URL does not exist!"
> ...
> URL does not exist!
>>>>
>
> A careful look at the difference between these two ought to make it
> clear what's going on.
>
> -[]z.
--
http://mail.python.org/mailman/listinfo/python-list
Help on instance of closeable_response in module Mechanize
Hi Python users, I have a question about the instance of closeable_response in module Mechanize. from mechanize import ParseResponse, urlopen url = "http://wwwsearch.sourceforge.net/mechanize/example.html"; r = urlopen(url) forms = ParseResponse(r, backwards_compat=False) html_lines = r.read() If I call ParseResponse() before r.read(), then lforms would be a list containing one form instance, and html_lines would be an empty string. If I call r.read() first, then html_lines would be the HTML source code of the page, but forms would be an empty list. Therefore, I have to open the url twice, once for each function, like this: r = urlopen(url) forms = ParseResponse(r, backwards_compat=False) r = urlopen(url) html_lines = r.read() I believe this shouldn't be necessary. What is the proper way of doing it? Thank you. - Yingjie -- http://mail.python.org/mailman/listinfo/python-list
Methods on file-like objects can only used once on one object?
Hi Python users,
I just realize that my post yesterday shouldn't be specifically for mechanize.
It should be a general question for file-like objects.
>>> f = open('my_file.txt')
>>> print f.readlines()
( prints a list of strings
>>> print f.readlines()
[]
There are quite a few methods for file-like objects that can only be used once
on one object. If I prefer to use some of these methods on one object, one
after another, like:
f.readlines()
f.read()
...
What should I do? Thank you.
- Yingjie
--
http://mail.python.org/mailman/listinfo/python-list
Python setup question
Hi, Sorry for this newbie question, I was wondering if anyone knows why when I run the test, the "test_anydbm" test will seg fault and the tests "test_aepack" and "test_al" are skipped? Thanks in advance. ______ Robert Lin Eng. Intern | Blue Jungle | Redwood City, CA [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Unbinding multiple variables
Hi! Is there a way to automate the unbinding of multiple variables? Say I have a list of the names of all variables in the current scope via dir(). Is there a command using del or something like that that will iterate the list and unbind each of the variables? Thanks much! (If anyone posts an answer, if you could also cc your reply to my email [EMAIL PROTECTED], would be much obliged. Thanks again!) Best, -Johnny www.johnny-lin.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Unbinding multiple variables
thanks everyone for the replies! John Hunter, yep, this is Johnny Lin in geosci :). re using return: the problem i have is somewhere in my code there's a memory leak. i realize return is supposed to unbind all the local variables, but since the memory leak is happening despite return, i thought it might help me track down the leak if i unbound everything explicitly that i had defined in local scope before i returned. or if anyone has recomm. on plugging leaks, would be thankful for any pointers there too. my understanding about locals() from the nutshell book was that i should treat that dictionary as read-only. is it safe to use it to delete entries? thanks again! -- http://mail.python.org/mailman/listinfo/python-list
Re: Unbinding multiple variables
thanks again for all the help! especially the advice on ideas of tracking down the memory leak :). (sorry for not mentioning it earlier...i had thought deleting everything might be a quick and dirty way short-term fix. :P) best, -Johnny -- http://mail.python.org/mailman/listinfo/python-list
Strange sqlite3 library behavior
Now I am now developing a program that base on sqlite3 in python.
But there is a strange problem.
That is, all data I insert into sqlite database do not goes into file
in disk.
It is really strange
Why do these data just keep in memory and discarded?
All things that really store in file is the table.
If I create a table, it would appears in the sqlite file.
This is the example :
import sqlite3
createSyntax = """CREATE TABLE IF NOT EXISTS category(
id INTEGER NOT NULL PRIMARY KEY
)"""
createSyntax2 = """CREATE TABLE IF NOT EXISTS category2(
id INTEGER NOT NULL PRIMARY KEY
)"""
insertSyntax = """INSERT INTO category VALUES (1234)"""
db = sqlite3.connect('test.db')
cur = db.cursor()
cur.execute(createSyntax)
cur.execute(insertSyntax)
# only by doing so can let inserted data store in file
cur.execute(createSyntax2)
cur.execute('select * from category')
print cur.fetchall()
db.close()
raw_input()
After I tried some way. I found that create table force data goes into
file. Otherwise all data would just keep in memory and discarded with
the program's ending.
Why sqlite3 just keep inserted data in memory? And how to force
inserted data into file?
Thanks.
Victor Lin.
--
http://mail.python.org/mailman/listinfo/python-list
Recursion limit of pickle?
Hi,
I encounter a problem with pickle.
I download a html from:
http://www.amazon.com/Magellan-Maestro-4040-Widescreen-Navigator/dp/B000NMKHW6/ref=sr_1_2?ie=UTF8&s=electronics&qid=1202541889&sr=1-2
and parse it with BeautifulSoup.
This page is very huge.
When I use pickle to dump it, a RuntimeError: maximum recursion depth
exceeded occur.
I think it is cause by this problem at first :
http://bugs.python.org/issue1757062
But and then I do not think so, because I log recursion call of pickle
in file
I found that the recursion limit is exceeded in mid-way to expand
whole the BeautifulSoup object.
Not repeat to call some methods.
This is the code for test.
from BeautifulSoup import *
import pickle as pickle
import urllib
doc = urllib.urlopen('http://www.amazon.com/Magellan-Maestro-4040-
Widescreen-Navigator/dp/B000NMKHW6/ref=sr_1_2?
ie=UTF8&s=electronics&qid=1202541889&sr=1-2')
import sys
sys.setrecursionlimit(4)
soup = BeautifulSoup(doc)
print pickle.dumps(soup)
---
What I want to ask is: Is this cause by the limit of recursion limit
and stack size?
I had tired cPickle at first, and then I try pickle, cPickle just stop
running program without any message.
I think it is also implement with recursion way, and it also over flow
stack when dumping soup.
Are there any version of pickle that implement with no-recursion way?
Thanks.
Victor Lin.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Recursion limit of pickle?
On 2月10日, 上午11時42分, "Gabriel Genellina" <[EMAIL PROTECTED]>
wrote:
> En Sat, 09 Feb 2008 09:49:46 -0200, Victor Lin <[EMAIL PROTECTED]>
> escribi�:
>
> > I encounter a problem with pickle.
> > I download a html from:
>
> >http://www.amazon.com/Magellan-Maestro-4040-Widescreen-Navigator/dp/B...
>
> > and parse it with BeautifulSoup.
> > This page is very huge.
> > When I use pickle to dump it, a RuntimeError: maximum recursion depth
> > exceeded occur.
>
> BeautifulSoup objects usually aren't pickleable, independently of your
> recursion error.
But I pickle and unpickle other soup objects successfully.
Only this object seems too deep to pickle.
>
> py> import pickle
> py> import BeautifulSoup
> py> soup = BeautifulSoup.BeautifulSoup("Hello, world!")
> py> print pickle.dumps(soup)
> Traceback (most recent call last):
> ...
> TypeError: 'NoneType' object is not callable
> py>
>
> Why do you want to pickle it? Store the downloaded page instead, and
> rebuild the BeautifulSoup object later when needed.
>
> --
> Gabriel Genellina
Because parsing html cost a lots of cpu time. So I want to cache soup
object as file. If I have to get same page, I can get it from cache
file, even the parsed soup file. My program's bottleneck is on parsing
html, so if I can parse once and unpickle them later, it could save a
lots of time.
--
http://mail.python.org/mailman/listinfo/python-list
Problem with import
Hi, I got some problem with import. I have some python file that should be executed. Something like unit-test or other small tool. The project is not so small. So I want to separate these files into different packages. Might like this: project/ run_task.py pkg1/ __init__.py run_lettle_task.py module_a.py module_b.py test1.py pkg2/ __init__.py module_c.py module_d.py test2.py Now here comes the problem: How to import these file correctly? For example: # file module_a.py - # This import force who run and import this file most in project/ this directory import pkg2.module_c # file test1.py # oops! If we run this file, # we got a error here, because this file is not in project/ import module_c # do some unittest - As you see, because I use import like this "import pkg2.module_c". This statement force python files which want to import and use this module should be in project. But I don't want to put all the files needed to execute in project. Because there are so many files and tests need to execute. If I put so many different things in project, to separate files to packages would be meaningless. What can I do? Actually, I know I can modify path to make python to search the project directory. Then in test1 or test2 I can import module_c. But here comes another problem: If I move my directory, I have to modify the path again. This is not a library, they are some programs to do specific task. I think they should be executable in every where I move or copy the directory to. Victor Lin. -- http://mail.python.org/mailman/listinfo/python-list
How to call python from a foreign language thread (C++)
Hi,
I am developing a program that use DirectShow to grab audio data from
media files. DirectShow use thread to pass audio data to the callback
function in my program, and I let that callback function call another
function in Python.
I use Boost.Python to wrapper my library, the callback function :
class PythonCallback {
private:
object m_Function;
public:
PythonCallback(object obj)
: m_Function(obj)
{}
void operator() (double time, const AudioData &data) {
// Call the callback function in python
m_Function(time, data);
}
};
Here comes the problem, a thread of DirectShow calls my
PythonCallback, namely, call the function in Python. Once it calls, my
program just crash. I found this should be threading problem. Then I
found this document:
http://docs.python.org/c-api/init.html
It seems that my program can't call to Python's function from thread
directly, because there is Global Interpreter Lock. The python's GIL
is so complex, I have no idea how it works. I'm sorry, what I can do
is to ask. My question is. What should I do before and after I call a
Python function from threads?
It may looks like this.
void operator() (double time, const AudioData &data) {
// acquire lock
m_Function(time, data);
// release lock
}
Thanks.
Victor Lin.
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to call python from a foreign language thread (C++)
On 2月4日, 上午2時44分, Philip Semanchuk wrote:
> On Feb 3, 2009, at 12:51 PM, Victor Lin wrote:
>
>
>
> > It seems that my program can't call to Python's function from thread
> > directly, because there is Global Interpreter Lock. The python's GIL
> > is so complex, I have no idea how it works. I'm sorry, what I can do
> > is to ask. My question is. What should I do before and after I call a
> > Python function from threads?
>
> Hi Victor,
> I asked a similar question yesterday and have gotten no response yet
> -- I hope you'll have better luck. I'm writing a C extension that
> wants to implement a callback in a new thread. Here's what I think
> I've learned from reading the threads API doc. Please note that this
> is a classic case of the blind leading the blind! I'm sure some (most?
> all?) of the ideas below are wrong, but I'm hoping that thinking
> through some of this "out loud" will help both of us. Or maybe some
> more knowledgeable person will take pity on/be appalled by my
> ignorance and come to our rescue. =)
>
> Python's infamous GIL doesn't exist when a program is single-threaded.
> Before a new thread is created, the main thread must call
> PyEval_InitThreads() to create the GIL. However, "It is not safe to
> call this function when it is unknown which thread (if any) currently
> has the global interpreter lock." Therefore my extension must do this:
>
> i_have_the_gil = 0;
> if (!PyEval_ThreadsInitialized()) {
> PyEval_InitThreads();
> /* "...when this function initializes the lock, it also acquires
> it." */
> i_have_the_gil = 1;
>
> }
>
> That ensures that the GIL is created.
>
> My extension will be calling from a newly-created C thread. The
> *Python* thread doesn't exist yet; so next I have to create it.
> Therefore --
>
> if (!i_have_the_gil)
> PyEval_AcquireLock();
>
> // Might not actually be the main thread but conceptually
> // it is OK to assume so here.
> main_thread = PyThreadState_Get();
>
> callback_thread = PyThreadState_New(main_thread->interp);
>
> PyThreadState_Swap(callback_thread);
>
> gstate = PyGILState_Ensure();
>
> call_callback_function();
>
> // Now unwind the above
>
> PyGILState_Release(gstate);
>
> PyThreadState_Swap(main_thread);
>
> PyThreadState_Clear(callback_thread);
>
> PyThreadState_Delete(callback_thread);
>
> PyEval_ReleaseLock();
>
> I haven't yet tested this! But hopefully it is the right idea and just
> needs a little fine tuning.
>
> I found this discussion
> useful:http://mail.python.org/pipermail/python-list/2006-November/413088.html
>
> It includes the quote, "The current thread state API doc, as you read
> it from top to bottom now, is in fact totally confusing for anyone who
> didn't develop Python himself"
>
> I like! =)
>
> Cheers
> Philip
Hi Philip,
It does not work. But however, thanks your help. I have tired so many
methods to do. But it
crash...crash..deadlock...deadlock..crash...crash... I have no any
tried success. I am going crazy. Could someone help me, thanks.
--
http://mail.python.org/mailman/listinfo/python-list
How to debug deadlock?
Hi, I am developing a multi-threading application, I encounter a deadlock. I use Visual C++ Express 2008 to trace the program. Once the deadlock occurs, I just pause the program and trace. I found that when deadlock occurs, there will be two threads called python from my C++ extension. All of them use Queue in python code, so I guess the deadlock might caused by Queue. But however, once the extension goes into python code, I can't see nothing but asm code and binary from the VC++ debugger. I would like to know are there any way to dump the call stack of python code after I paused the program? And how can I know what lock are there in threads caused the deadlock? Thanks. Victor Lin. -- http://mail.python.org/mailman/listinfo/python-list
Callback from c thread with ctypes
Hi, I am going to develop a c library binding with ctypes. That c library will call callback from worker threads it created. Here comes the problem : Will the GIL be acquired before it goes into Python function? I got a little try.. DSPPROC = WINFUNCTYPE(None, DWORD, DWORD, c_void_p, DWORD, c_void_p) def test(handle, channel, buffer, length, user): print handle, channel, buffer, length, user dsp = BASS_ChannelSetDSP(stream, DSPPROC(test), None, 123) I got "access violation" when I run it... It seems that the ctypes did't acquire GIL before it call the python callback. As the document says. WINFUNCTYPE will release GIL during the call But it does not mention callback about Python function? How about a call from another thread? Could somebody help me? Thanks. Victor Lin. -- http://mail.python.org/mailman/listinfo/python-list
Re: Callback from c thread with ctypes
On 3月8日, 下午9時56分, "Diez B. Roggisch" wrote: > Victor Lin schrieb: > > > > > Hi, > > > I am going to develop a c library binding with ctypes. That c library > > will call callback from worker threads it created. Here comes the > > problem : Will the GIL be acquired before it goes into Python > > function? > > > I got a little try.. > > > DSPPROC = WINFUNCTYPE(None, DWORD, DWORD, c_void_p, DWORD, c_void_p) > > > def test(handle, channel, buffer, length, user): > > print handle, channel, buffer, length, user > > > dsp = BASS_ChannelSetDSP(stream, DSPPROC(test), None, 123) > > > I got "access violation" when I run it... It seems that the ctypes > > did't acquire GIL before it call the python callback. As the document > > says. > > > WINFUNCTYPE will release GIL during the call > > > But it does not mention callback about Python function? How about a > > call from another thread? Could somebody help me? > > The releasing takes only place when entering a c-function. A > python-callback acquires the GIL, see callbacks.c in the python source. > > The access violation has nothing to do with that I presume, that's just > a general programming error as it happens with ctypes during development. > > Diez Hi, Thanks your replying, I try it again, found that my program only crash when I call it from python's IDLE. If I click it, namely execute it with python directly, it works fine. Why the program will crash within IDLE? -- http://mail.python.org/mailman/listinfo/python-list
Re: Callback from c thread with ctypes
On 3月8日, 下午10時20分, Christian Heimes wrote: > Victor Lin wrote: > > Hi, > > > I am going to develop a c library binding with ctypes. That c library > > will call callback from worker threads it created. Here comes the > > problem : Will the GIL be acquired before it goes into Python > > function? > > > I got a little try.. > > > DSPPROC = WINFUNCTYPE(None, DWORD, DWORD, c_void_p, DWORD, c_void_p) > > > def test(handle, channel, buffer, length, user): > > print handle, channel, buffer, length, user > > > dsp = BASS_ChannelSetDSP(stream, DSPPROC(test), None, 123) > > > I got "access violation" when I run it... It seems that the ctypes > > did't acquire GIL before it call the python callback. As the document > > says. > > > WINFUNCTYPE will release GIL during the call > > > But it does not mention callback about Python function? How about a > > call from another thread? Could somebody help me? > > You can't call Python code from an arbitrary thread. Before you are > allowed to call a Python function from a thread, you must enable > Python's threading subsystem and register the thread. Python need to > store information (like the exception information) in a thread local > variable (TLS). > > I can't tell you how to register your thread with Python, though. > > Christian I know I have to call PyEval_InitThreads if my module create threads that will contact python stuff, for example, call a python callback function from threads. But however, it is ctypes. I have no idea should I do that for the imported dll? If it is, how? Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Any c header parser for generate ctypes module?
Hi, I am writing python binding for some c library. It is really a super boring job. Copy... paste... modify.. copy paste...modify I am wondering, I am a programmer, why I can't do this job like a programmer? So I think the best way to write binding for those c libraries, is to write a parser that parse header of c library and generate ctypes python module automatically. My question is, is there any available tools can achieve this? If not, what tool can I use to such a job easily. I need a c parser, is there any C parser written in python? Thanks. Victor Lin. -- http://mail.python.org/mailman/listinfo/python-list
How to redirect operation methods to some sepcific method easily?
Hi,
I'd like to write some class that can help me build reusable formula
easily, some simple code like this.
# -*- coding: utf8 -*-
class OperationResult:
def __init__(self, left, right):
self.dataSource = dataSource
def __add__(self, other):
self.dataSource.stack.append('+')
return self
def __div__(self, other):
self.dataSource.stack.append('/')
return self
class Operand:
def __init__(self, dataSource, name):
self.dataSource = dataSource
self.name = name
def __add__(self, other):
self.dataSource.stack.append(self)
self.dataSource.stack.append(other)
self.dataSource.stack.append('+')
return OperationResult(self.dataSource)
class DataSource:
def __init__(self):
self.stack = []
def __getitem__(self, key):
return Operand(self, key)
def ROE(dataSource):
a = dataSource[u'股東權益總額']
b = dataSource[u'每股淨額']
c = dataSource[u'當季平均市價']
return (a + b) / c
Now I can use ROE formula easily, it will return a object that contain
formula stack like this
[a, b, +, c, /]
And then I can get real data of those items from database or other
place. The formula depends on nothing. It don't care where to get data
and how to calculate it. I can reuse it easily.
Now, here comes the problem : I have to override all the operation
methods, such as __add__ and __mul__.
I know the most stupid way is just to write all of them like this.
def __add__(self, other):
self.leftOperation('add', other)
def __mul__(self, other):
self.leftOperation('mul', other)
But I don't want to do that in this stupid way. I want to use little
code to redirect these operation methods to some specific method with
it's name.
What can I do?
Thanks.
--
http://mail.python.org/mailman/listinfo/python-list
Logging library unicode problem
Hi,
I'm writting a application using python standard logging system. I
encounter some problem with unicode message passed to logging library.
I found that unicode message will be messed up by logging handler.
piese of StreamHandler:
try:
self.stream.write(fs % msg)
except UnicodeError:
self.stream.write(fs % msg.encode("UTF-8"))
It just write the message to stream. If there is some unicode error,
it would rewrite msg with utf8 encoding.
I write some code to try:
import sys
print u'中文字測試'
print sys.stdout.encoding
sys.stdout.write(u'中文')
result of that program:
中文字測試
cp950
Traceback (most recent call last):
File "update_stockprice.py", line 92, in
sys.stdout.write(u'銝剜?')
UnicodeEncodeError: 'ascii' codec can't encode characters in position
0-1: ordin
al not in range(128)
It show that
1. print statement encode what it get with stream.encoding?
2. stream.write don't do anything like encoding, just write it
(because it might be binary data?)
So the problem is : the StreamHandler of standard logging library use
stream.write to log message, if there is unicode error, unicode string
will be encode to utf8. This behavior mess my unicode up.
Here I modify the code of StreamHandler:
try:
print >> self.stream, msg
#self.stream.write(fs % msg)
except UnicodeError:
self.stream.write(fs % msg.encode("UTF-8"))
I replace stream.write with print statement, so that it will try to
use stream.encoding to encode msg. Now everything works fine.
My question is :
Could the behavior of StreamHandler be considered as a bug?
If it is, how to report this bug?
Is my solution correct?
Are there any side effect will caused by doing so?
If the code I write is fine, and solve that problem, how to report it
to Python's project?
I think this could be helpful for people who also encountered this
problem.
Thanks.
Victor Lin.
--
http://mail.python.org/mailman/listinfo/python-list
Ask how to use HTMLParser
I am a new guy to use Python, but I want to parse a html page now. I tried to use HTMLParse. Here is my sample code: -- from HTMLParser import HTMLParser from urllib2 import urlopen class MyParser(HTMLParser): title = "" is_title = "" def __init__(self, url): HTMLParser.__init__(self) req = urlopen(url) self.feed(req.read()) def handle_starttag(self, tag, attrs): if tag == 'div' and attrs[0][1] == 'articleTitle': print "Found link => %s" % attrs[0][1] self.is_title = 1 def handle_data(self, data): if self.is_title: print "here" self.title = data print self.title self.is_title = 0 --- For the tag --- open article title --- I use my code to parse it. I can locate the div tag but I don't know how to get the text for the tag which is "open article title" in my example. How can I get the html content? What's wrong in my handle_data function? Thanks Water Lin -- Water Lin's notes and pencils: http://en.waterlin.org Email: [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: Ask how to use HTMLParser
h0uk writes: > On 8 янв, 08:44, Water Lin wrote: >> I am a new guy to use Python, but I want to parse a html page now. I >> tried to use HTMLParse. Here is my sample code: >> -- >> from HTMLParser import HTMLParser >> from urllib2 import urlopen >> >> class MyParser(HTMLParser): >> title = "" >> is_title = "" >> def __init__(self, url): >> HTMLParser.__init__(self) >> req = urlopen(url) >> self.feed(req.read()) >> >> def handle_starttag(self, tag, attrs): >> if tag == 'div' and attrs[0][1] == 'articleTitle': >> print "Found link => %s" % attrs[0][1] >> self.is_title = 1 >> >> def handle_data(self, data): >> if self.is_title: >> print "here" >> self.title = data >> print self.title >> self.is_title = 0 >> --- >> >> For the tag >> --- >> open article title >> --- >> >> I use my code to parse it. I can locate the div tag but I don't know how >> to get the text for the tag which is "open article title" in my example. >> >> How can I get the html content? What's wrong in my handle_data function? >> >> Thanks >> >> Water Lin >> >> -- >> Water Lin's notes and pencils:http://en.waterlin.org >> Email: [email protected] > > I want to say your code works well But in handle_data I can't print self.title. I don't why I can't set the self.title in handle_data. Thanks Water Lin -- Water Lin's notes and pencils: http://en.waterlin.org Email: [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Memory usage problem of twisted server
Hi, I encountered an increasing memory usage problem of my twisted server. I have posted a question on stackoverflow: http://stackoverflow.com/questions/2100192/how-to-find-the-source-of-increasing-memory-usage-of-a-twisted-server I have read the article "Improving Python's Memory Allocator" ( http://evanjones.ca/memoryallocator/ ) and Python Memory Management ( http://evanjones.ca/python-memory.html ). And I now know little about how Python manages memory. I am wondering, is that the increasing memory usage problem of my audio broadcasting caused by the how python manage memory? In my server, there are lots of audio data chunks, they are not all in fixed size. For example, the size of audio stream might looks like this: ... chunk size 822 chunk size 878 chunk size 1690 chunk size 1659 chunk size 1643 chunk size 1952 chunk size 373 chunk size 763 chunk size 1535 chunk size 1665 ... All of those chunks are str object. The size of chunks depend on mp3 encoder. You can see most the size of those chunks is bigger than 256 bytes, so that Python uses malloc to allocate spaces for those chunks. In my mind, I saw those chunks occupy different place in heap, all of them are not in fixed size, so that even when the chunk is freed, it is difficult for malloc to find the right place for the new chunk. As the result, there are so many chunks placed in large range of heap, it is difficult for OS to swap pages out. Then the RSS is always high. Is that my guessing correct? How can I monitor the memory allocation of Python? Is there any available tools for this? Or is that modifying the Python source code and recompiling the only way to monitor allocation of memory? Thanks. Victor Lin. -- http://mail.python.org/mailman/listinfo/python-list
A tool for find dependencies relationships behind Python projects
Hi, I just wrote a tool for drawing dependencies relationships diagram of Python project on Pypi. Here is the home page of the tool: http://code.google.com/p/python-gluttony/ Some examples: Sprox: http://static.ez2learn.com/gluttony/sprox_dot.png TurboGears2: http://static.ez2learn.com/gluttony/tg2_dot.png Hope this could be helpful :P Regards. Victor Lin. -- http://mail.python.org/mailman/listinfo/python-list
Re: A tool for find dependencies relationships behind Python projects
On 2月23日, 上午12時32分, Hellmut Weber wrote:
> Hi Victor,
> I would be intereseted to use your tool ;-)
>
> My system is Sabayon-5.1 on Lenovo T61.
> Trying for the first time easy_install I get the following error:
>
>
>
> r...@sylvester ~ # easy_install gluttony
> Searching for gluttony
> Readinghttp://pypi.python.org/simple/gluttony/
> Readinghttp://code.google.com/p/python-gluttony/
> Best match: Gluttony 0.3
>
> Downloadinghttp://pypi.python.org/packages/source/G/Gluttony/Gluttony-0.3.zip#md...
>
> Processing Gluttony-0.3.zip
> Running Gluttony-0.3/setup.py -q bdist_egg --dist-dir
> /tmp/easy_install-uPz7qO/Gluttony-0.3/egg-dist-tmp-CJI_LD
> Traceback (most recent call last):
> File "/usr/bin/easy_install", line 9, in
> load_entry_point('distribute==0.6.8', 'console_scripts',
> 'easy_install')()
> File
> "/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py",
> line 1708, in main
> with_ei_usage(lambda:
> File
> "/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py",
> line 1696, in with_ei_usage
> return f()
> File
> "/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py",
> line 1712, in
> distclass=DistributionWithoutHelpCommands, **kw
> File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup
> dist.run_commands()
> File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands
> self.run_command(cmd)
> File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command
> cmd_obj.run()
> File
> "/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py",
> line 236, in run
> self.easy_install(spec, not self.no_deps)
> File
> "/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py",
> line 471, in easy_install
> return self.install_item(spec, dist.location, tmpdir, deps)
> File
> "/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py",
> line 501, in install_item
> dists = self.install_eggs(spec, download, tmpdir)
> File
> "/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py",
> line 680, in install_eggs
> return self.build_and_install(setup_script, setup_base)
> File
> "/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py",
> line 957, in build_and_install
> self.run_setup(setup_script, setup_base, args)
> File
> "/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py",
> line 946, in run_setup
> run_setup(setup_script, args)
> File "/usr/lib64/python2.6/site-packages/setuptools/sandbox.py", line
> 29, in run_setup
> lambda: execfile(
> File "/usr/lib64/python2.6/site-packages/setuptools/sandbox.py", line
> 70, in run
> return func()
> File "/usr/lib64/python2.6/site-packages/setuptools/sandbox.py", line
> 31, in
> {'__file__':setup_script, '__name__':'__main__'}
> File "setup.py", line 9, in
> File "/tmp/easy_install-uPz7qO/Gluttony-0.3/gluttony/__init__.py",
> line 1, in
> #
> File "/tmp/easy_install-uPz7qO/Gluttony-0.3/gluttony/gluttony.py",
> line 13, in
> ImportError: No module named pip.log
> r...@sylvester ~ #
>
>
>
> I emerged app-misc/pip, but that didn't help, the error remains the same
>
> What is missing?
>
> Any help appreciated
>
> Best regards
>
> Hellmut
>
> Am 19.02.2010 17:16, schrieb Victor Lin:
>
>
>
>
>
> > Hi,
>
> > I just wrote a tool for drawing dependencies relationships diagram of
> > Python project on Pypi. Here is the home page of the tool:
>
> >http://code.google.com/p/python-gluttony/
>
> > Some examples:
> > Sprox:
> >http://static.ez2learn.com/gluttony/sprox_dot.png
>
> > TurboGears2:
> >http://static.ez2learn.com/gluttony/tg2_dot.png
>
> > Hope this could be helpful :P
>
> > Regards.
> > Victor Lin.
>
> --
> Dr. Hellmut Weber [email protected]
> Degenfeldstraße 2 tel +49-89-3081172
> D-80803 München-Schwabing mobil +49-172-8450321
> please: No DOCs, no PPTs. why: tinyurl.com/cbgq
Hi,
That is a mistake I made in when I am making the distribute. Thanks
your reporting. I have already fixed the problem and released
Gluttony 0.4, and I tried to install that, it works fine.
Victor Lin.
--
http://mail.python.org/mailman/listinfo/python-list
python cgi webpage won't redirect before background children processes finish
Hi all,
I have a python cgi script which looks like this:
[CODE STARTING HERE]
open('x')
print 'Content-Type: text/html\n\n'
..
print '' % myURL
..
### after printing the webpage
os.system('python myfile.py')
logfile.write('END OF SCRIPT')
logfile.close()
[CODE ENDING]
Question: I want this cgi webpage to automatically redirect to myURL after 15
seconds.
As soon as the execution of this script finishes, the corresponding job
disappears from
ps command output, the webpage is displayed and the log file is written.
However, the
webpage doesn't redirect after 15 seconds unless the background child process
'python myfile.py' finishes by then.
Is this problem caused by initialing the children job through os.system() ?
Does any one
know what I need to change to make the redirection work before background
children processes finish?
Thank you very much!
Yingjie
--
http://mail.python.org/mailman/listinfo/python-list
py-cocoa?
It's known that combining GTK+ or Qt with Python could enable the GUI design with pygtk/pyqt. In Mac OSX, it's suggested that use Cocoa be the GUI framework. Is there py-cocoa framework? -- http://mail.python.org/mailman/listinfo/python-list
