Peter Otten wrote:
> Like Gerhard says, in the long run you are probably better off with
> ElementTree.
In the long run it's even better to use lxml [1]. It's the fastest und
most powerful XML library for Python. It also supports element tree.
Christian
[1] http://codespeak.net/lxml/
--
http://
Thanks for the help.
I converted everything into the StringIO() format. Memory is still getting
chewed up. I will look at ElementTree later but for now I believe the speed
issue must be related to the amount of memory that is getting used. It is
causing all of windows to slow to a crawl. gc.coll
Carbon Man wrote:
> Very new to Python, running 2.5 on windows.
> I am processing an XML file (7.2MB). Using the standard library I am
> recursively processing each node and parsing it. The branches don't go
> particularly deep. What is happening is that the program is running really
> really slow
Here's a link for you:
http://wiki.python.org/moin/PythonSpeed/PerformanceTips
which also talks about string concatenation and othere do's and don'ts.
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Carbon Man wrote:
> Very new to Python, running 2.5 on windows.
> I am processing an XML file (7.2MB). Using the standard library I am
> recursively processing each node and parsing it. The branches don't go
> particularly deep. What is happening is that the program is running really
> really sl
Ken> Unfortunately, Python has some problems in this area. In
Ken> particular, since ubiquitous lists and dictionaries are dynamically
Ken> resized as needed, memory fragmentation seems inevitable.
That's not necessarily true. Also, I would say that Python has made
tradeoffs in this
Thanks Marc,
I just tried shelve but it is very slow :(
I haven't tried the dbs yet.
Andre
Marc 'BlackJack' Rintsch a écrit :
> On Mon, 15 Oct 2007 11:31:59 +0200, amdescombes wrote:
>
>> Are there any classes that implement disk based dictionaries?
>
> Take a look at the `shelve` module from
On Mon, 15 Oct 2007 11:31:59 +0200, amdescombes wrote:
> Are there any classes that implement disk based dictionaries?
Take a look at the `shelve` module from the standard library.
Or object databases like ZODB or Durus.
Ciao,
Marc 'BlackJack' Rintsch
--
http://mail.python.org/mailman/
Yes, I think that might be the issue, perhaps I could implement the
solution using several dictionaries instead of just one.
Are there any classes that implement disk based dictionaries?
Thanks,
Andre
>
> I don't know whether Python dictionaries must live in a contiguous piece of
> memory, but
AMD <[EMAIL PROTECTED]> wrote:
>
>I do the reading one line at a time, the problem seems to be with the
>dictionary I am creating.
I don't know whether Python dictionaries must live in a contiguous piece of
memory, but if so, that could be the issue. The system DLLs in Server 2003
have been "reb
Hi Brad,
I do the reading one line at a time, the problem seems to be with the
dictionary I am creating.
Andre
> amdescombes wrote:
>> Hi,
>>
>> I am using Python 2.5.1
>> I have an application that reads a file and generates a key in a
>> dictionary for each line it reads. I have managed to r
amdescombes wrote:
> Hi,
>
> I am using Python 2.5.1
> I have an application that reads a file and generates a key in a
> dictionary for each line it reads. I have managed to read a 1GB file and
> generate more than 8 million keys on an Windows XP machine with only 1GB
> of memory and all works
amdescombes wrote:
> Hi,
>
> I am using Python 2.5.1
> I have an application that reads a file and generates a key in a
> dictionary for each line it reads. I have managed to read a 1GB file and
> generate more than 8 million keys on an Windows XP machine with only 1GB
> of memory and all works
13 matches
Mail list logo
