Fredrik Lundh wrote:
> Christopher Subich wrote:
>>
>>I have access to an itanium system with a metric ton of memory. I
>>-think- that the Python version is still only a 32-bit python
>
>
> an ILP64 system is a system where int, long, and pointer are all 64 bits,
> so a 32-bit python on a 64-bi
Christopher Subich wrote:
>> anyone out there with an ILP64 system?
>
> I have access to an itanium system with a metric ton of memory. I
> -think- that the Python version is still only a 32-bit python
an ILP64 system is a system where int, long, and pointer are all 64 bits,
so a 32-bit python o
Fredrik Lundh wrote:
> Harald Karner wrote:
>>>python -c "print len('m' * ((2048*1024*1024)-1))"
>>
>>2147483647
>
>
> the string type uses the ob_size field to hold the string length, and
> ob_size is an integer:
>
> $ more Include/object.h
> ...
> int ob_size; /* Number of items in va
"Gerald Klix" <[EMAIL PROTECTED]> schrieb im Newsbeitrag
news:[EMAIL PROTECTED]
> Did you consider the mmap library?
> Perhaps it is possible to avoid to hold these big stings in memory.
> BTW: AFAIK it is not possible in 32bit windows for an ordinary programm
> to allocate more than 2 GB. That re
Claudio Grondi <[EMAIL PROTECTED]> wrote:
...
> In this context I am very curious how many of such
> 2 GByte strings is it possible to create within a
> single Python process?
VM (Virtual Memory) may make the issue difficult to answer precisely.
With a Python build for 64-bit addressing (and r
"Harald Karner" <[EMAIL PROTECTED]> schrieb im Newsbeitrag
news:[EMAIL PROTECTED]
> Claudio Grondi wrote:
> > Anyone on a big Linux machine able to do e.g. :
> > \>python -c "print len('m' * 2500*1024*1024)"
> > or even more without a memory error?
>
> I tried on a Sun with 16GB Ram (Python 2.3.
Harald Karner wrote:
> I tried on a Sun with 16GB Ram (Python 2.3.2)
> seems like 2GB is the limit for string size:
>
> > python -c "print len('m' * 2048*1024*1024)"
> Traceback (most recent call last):
> File "", line 1, in ?
> OverflowError: repeated string is too long
>
> > python -c "print l
Claudio Grondi wrote:
> Anyone on a big Linux machine able to do e.g. :
> \>python -c "print len('m' * 2500*1024*1024)"
> or even more without a memory error?
I tried on a Sun with 16GB Ram (Python 2.3.2)
seems like 2GB is the limit for string size:
> python -c "print len('m' * 2048*1024*1024)
I was also able to create a 1GB string on a different system (Linux 2.4.x,
32-bit Dual Intel Xeon, 8GB RAM, python 2.2).
$ python -c 'print len("m" * 1024*1024*1024)'
1073741824
I agree with another poster that you may be hitting Windows limitations
rather
than Python ones, but I am certainly not f
Christophe wrote:
> > Did you consider the mmap library?
> > Perhaps it is possible to avoid to hold these big stings in memory.
> > BTW: AFAIK it is not possible in 32bit windows for an ordinary programm
> > to allocate more than 2 GB. That restriction comes from the jurrasic
> > MIPS-Processors,
Gerald Klix a écrit :
> Did you consider the mmap library?
> Perhaps it is possible to avoid to hold these big stings in memory.
> BTW: AFAIK it is not possible in 32bit windows for an ordinary programm
> to allocate more than 2 GB. That restriction comes from the jurrasic
> MIPS-Processors, that
time
>>> bz2 fails to compress running endlessly consuming 99% of CPU time
>>>
>>>The same works with a 10 MByte string without any problem.
>>>
>>>So what? Is there no compression support for large sized strings in
>
> Python?
>
>>you'r
gt; >
> > At least on my system:
> > zlib fails to decompress raising a memory error
> > pylzma fails to decompress running endlessly consuming 99% of CPU time
> > bz2 fails to compress running endlessly consuming 99% of CPU time
> >
> > The same works with a 1
On this system (Linux 2.6.x, AMD64, 2 GB RAM, python2.4) I am able to
construct a 1 GB string by repetition, as well as compress a 512MB
string with gzip in one gulp.
$ cat claudio.py
s = '1234567890'*(1048576*50)
import zlib
c = zlib.compress(s)
print len(c)
open("/tmp/c
U time
> bz2 fails to compress running endlessly consuming 99% of CPU time
>
> The same works with a 10 MByte string without any problem.
>
> So what? Is there no compression support for large sized strings in Python?
you're probably measuring windows' memory managment rath
works with a 10 MByte string without any problem.
So what? Is there no compression support for large sized strings in Python?
Am I doing something the wrong way here?
Is there any and if yes, what is the theoretical upper limit of string size
which can be processed by each of the compression
16 matches
Mail list logo
